Note: This is test shard 1 of 8.
[==========] Running 8 tests from 6 test suites.
[----------] Global test environment set-up.
[----------] 1 test from AdminCliTest
[ RUN ] AdminCliTest.RebalancerReportOnly
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20240502 20:45:25.235836 22709 test_util.cc:268] Using random seed: -1363313557
I20240502 20:45:25.242133 22709 ts_itest-base.cc:112] Starting cluster with:
I20240502 20:45:25.242249 22709 ts_itest-base.cc:113] --------------
I20240502 20:45:25.242285 22709 ts_itest-base.cc:114] 5 tablet servers
I20240502 20:45:25.242305 22709 ts_itest-base.cc:115] 3 replicas per TS
I20240502 20:45:25.242317 22709 ts_itest-base.cc:116] --------------
2024-05-02T20:45:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T20:45:25Z Disabled control of system clock
I20240502 20:45:25.250766 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.45.126:46835
--webserver_interface=127.22.45.126
--webserver_port=0
--builtin_ntp_servers=127.22.45.84:39155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.45.126:46835 with env {}
W20240502 20:45:25.351159 22721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:25.351356 22721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:25.351392 22721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:25.354301 22721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 20:45:25.354348 22721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:25.354367 22721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 20:45:25.354384 22721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 20:45:25.357867 22721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:39155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.45.126:46835
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.45.126:46835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.22.45.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.22721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:25.358639 22721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:25.359658 22721 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:25.366369 22729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:25.366642 22726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:25.367039 22727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:25.367380 22721 server_base.cc:1034] running on GCE node
I20240502 20:45:25.367811 22721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:25.368667 22721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:25.369809 22721 hybrid_clock.cc:648] HybridClock initialized: now 1714682725369790 us; error 34 us; skew 500 ppm
I20240502 20:45:25.376005 22721 webserver.cc:460] Webserver started at http://127.22.45.126:42927/ using document root <none> and password file <none>
I20240502 20:45:25.376544 22721 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:25.376605 22721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:25.376780 22721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:25.378440 22721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "356892dd58e74d44bd66600717c899e0"
format_stamp: "Formatted at 2024-05-02 20:45:25 on dist-test-slave-g7g5"
I20240502 20:45:25.378928 22721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "356892dd58e74d44bd66600717c899e0"
format_stamp: "Formatted at 2024-05-02 20:45:25 on dist-test-slave-g7g5"
I20240502 20:45:25.381992 22721 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240502 20:45:25.384021 22735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:25.384981 22721 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20240502 20:45:25.385092 22721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal
uuid: "356892dd58e74d44bd66600717c899e0"
format_stamp: "Formatted at 2024-05-02 20:45:25 on dist-test-slave-g7g5"
I20240502 20:45:25.385172 22721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:25.405718 22721 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:25.406349 22721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:25.406524 22721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:25.412668 22721 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.126:46835
I20240502 20:45:25.412673 22787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.126:46835 every 8 connection(s)
I20240502 20:45:25.413596 22721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 20:45:25.415186 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 22721
I20240502 20:45:25.415304 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 20:45:25.416036 22788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:25.421177 22788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 356892dd58e74d44bd66600717c899e0: Bootstrap starting.
I20240502 20:45:25.423115 22788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 356892dd58e74d44bd66600717c899e0: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:25.423738 22788 log.cc:824] T 00000000000000000000000000000000 P 356892dd58e74d44bd66600717c899e0: Log is configured to *not* fsync() on all Append() calls
I20240502 20:45:25.425197 22788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 356892dd58e74d44bd66600717c899e0: No bootstrap required, opened a new log
I20240502 20:45:25.427719 22788 raft_consensus.cc:348] T 00000000000000000000000000000000 P 356892dd58e74d44bd66600717c899e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "356892dd58e74d44bd66600717c899e0" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 46835 } }
I20240502 20:45:25.427898 22788 raft_consensus.cc:374] T 00000000000000000000000000000000 P 356892dd58e74d44bd66600717c899e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:25.427963 22788 raft_consensus.cc:729] T 00000000000000000000000000000000 P 356892dd58e74d44bd66600717c899e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 356892dd58e74d44bd66600717c899e0, State: Initialized, Role: FOLLOWER
I20240502 20:45:25.428359 22788 consensus_queue.cc:260] T 00000000000000000000000000000000 P 356892dd58e74d44bd66600717c899e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "356892dd58e74d44bd66600717c899e0" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 46835 } }
I20240502 20:45:25.428484 22788 raft_consensus.cc:388] T 00000000000000000000000000000000 P 356892dd58e74d44bd66600717c899e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 20:45:25.428546 22788 raft_consensus.cc:482] T 00000000000000000000000000000000 P 356892dd58e74d44bd66600717c899e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 20:45:25.428683 22788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 356892dd58e74d44bd66600717c899e0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:25.429446 22788 raft_consensus.cc:504] T 00000000000000000000000000000000 P 356892dd58e74d44bd66600717c899e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "356892dd58e74d44bd66600717c899e0" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 46835 } }
I20240502 20:45:25.429731 22788 leader_election.cc:302] T 00000000000000000000000000000000 P 356892dd58e74d44bd66600717c899e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 356892dd58e74d44bd66600717c899e0; no voters:
I20240502 20:45:25.429951 22788 leader_election.cc:288] T 00000000000000000000000000000000 P 356892dd58e74d44bd66600717c899e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 20:45:25.430258 22793 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 356892dd58e74d44bd66600717c899e0 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:45:25.430436 22793 raft_consensus.cc:686] T 00000000000000000000000000000000 P 356892dd58e74d44bd66600717c899e0 [term 1 LEADER]: Becoming Leader. State: Replica: 356892dd58e74d44bd66600717c899e0, State: Running, Role: LEADER
I20240502 20:45:25.430701 22793 consensus_queue.cc:237] T 00000000000000000000000000000000 P 356892dd58e74d44bd66600717c899e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "356892dd58e74d44bd66600717c899e0" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 46835 } }
I20240502 20:45:25.430955 22788 sys_catalog.cc:564] T 00000000000000000000000000000000 P 356892dd58e74d44bd66600717c899e0 [sys.catalog]: configured and running, proceeding with master startup.
I20240502 20:45:25.431946 22794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 356892dd58e74d44bd66600717c899e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "356892dd58e74d44bd66600717c899e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "356892dd58e74d44bd66600717c899e0" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 46835 } } }
I20240502 20:45:25.432050 22794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 356892dd58e74d44bd66600717c899e0 [sys.catalog]: This master's current role is: LEADER
I20240502 20:45:25.432319 22795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 356892dd58e74d44bd66600717c899e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 356892dd58e74d44bd66600717c899e0. Latest consensus state: current_term: 1 leader_uuid: "356892dd58e74d44bd66600717c899e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "356892dd58e74d44bd66600717c899e0" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 46835 } } }
I20240502 20:45:25.432401 22802 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 20:45:25.432435 22795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 356892dd58e74d44bd66600717c899e0 [sys.catalog]: This master's current role is: LEADER
I20240502 20:45:25.434971 22802 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 20:45:25.439648 22802 catalog_manager.cc:1347] Generated new cluster ID: ca8fafa8349e431eb49e9fa7acf443b5
I20240502 20:45:25.439713 22802 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 20:45:25.452852 22802 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 20:45:25.453671 22802 catalog_manager.cc:1504] Loading token signing keys...
I20240502 20:45:25.468288 22802 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 356892dd58e74d44bd66600717c899e0: Generated new TSK 0
I20240502 20:45:25.468971 22802 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 20:45:25.482590 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.65:0
--local_ip_for_outbound_sockets=127.22.45.65
--webserver_interface=127.22.45.65
--webserver_port=0
--tserver_master_addrs=127.22.45.126:46835
--builtin_ntp_servers=127.22.45.84:39155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20240502 20:45:25.585384 22812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:25.585578 22812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:25.585618 22812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:25.588706 22812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:25.588810 22812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.65
I20240502 20:45:25.592326 22812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:39155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.22.45.65
--webserver_port=0
--tserver_master_addrs=127.22.45.126:46835
--never_fsync=true
--heap_profile_path=/tmp/kudu.22812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.65
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:25.593128 22812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:25.594174 22812 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:25.601636 22818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:25.602008 22820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:25.602361 22817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:25.602780 22812 server_base.cc:1034] running on GCE node
I20240502 20:45:25.603087 22812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:25.603569 22812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:25.604717 22812 hybrid_clock.cc:648] HybridClock initialized: now 1714682725604704 us; error 26 us; skew 500 ppm
I20240502 20:45:25.606433 22812 webserver.cc:460] Webserver started at http://127.22.45.65:42637/ using document root <none> and password file <none>
I20240502 20:45:25.606962 22812 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:25.607019 22812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:25.607203 22812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:25.608729 22812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "10c79541a38f410893c3b6474f9d0dae"
format_stamp: "Formatted at 2024-05-02 20:45:25 on dist-test-slave-g7g5"
I20240502 20:45:25.609170 22812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "10c79541a38f410893c3b6474f9d0dae"
format_stamp: "Formatted at 2024-05-02 20:45:25 on dist-test-slave-g7g5"
I20240502 20:45:25.612201 22812 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 20:45:25.614122 22826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:25.615053 22812 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:45:25.615157 22812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "10c79541a38f410893c3b6474f9d0dae"
format_stamp: "Formatted at 2024-05-02 20:45:25 on dist-test-slave-g7g5"
I20240502 20:45:25.615232 22812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:25.634624 22812 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:25.635231 22812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:25.635385 22812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:25.635907 22812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:25.636760 22812 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:25.636806 22812 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:25.636857 22812 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:25.636883 22812 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:25.645618 22812 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.65:33845
I20240502 20:45:25.645641 22939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.65:33845 every 8 connection(s)
I20240502 20:45:25.646559 22812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 20:45:25.647877 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 22812
I20240502 20:45:25.648005 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 20:45:25.650221 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.66:0
--local_ip_for_outbound_sockets=127.22.45.66
--webserver_interface=127.22.45.66
--webserver_port=0
--tserver_master_addrs=127.22.45.126:46835
--builtin_ntp_servers=127.22.45.84:39155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20240502 20:45:25.656055 22940 heartbeater.cc:347] Connected to a master server at 127.22.45.126:46835
I20240502 20:45:25.656271 22940 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:25.656788 22940 heartbeater.cc:511] Master 127.22.45.126:46835 requested a full tablet report, sending...
I20240502 20:45:25.658306 22752 ts_manager.cc:194] Registered new tserver with Master: 10c79541a38f410893c3b6474f9d0dae (127.22.45.65:33845)
I20240502 20:45:25.659458 22752 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.65:47531
W20240502 20:45:25.754158 22943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:25.754351 22943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:25.754407 22943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:25.757339 22943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:25.757452 22943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.66
I20240502 20:45:25.760917 22943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:39155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.22.45.66
--webserver_port=0
--tserver_master_addrs=127.22.45.126:46835
--never_fsync=true
--heap_profile_path=/tmp/kudu.22943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.66
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:25.761718 22943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:25.762785 22943 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:25.770298 22948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:25.770640 22951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:25.770890 22949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:25.771287 22943 server_base.cc:1034] running on GCE node
I20240502 20:45:25.771600 22943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:25.772092 22943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:25.773238 22943 hybrid_clock.cc:648] HybridClock initialized: now 1714682725773225 us; error 26 us; skew 500 ppm
I20240502 20:45:25.774899 22943 webserver.cc:460] Webserver started at http://127.22.45.66:34527/ using document root <none> and password file <none>
I20240502 20:45:25.775403 22943 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:25.775456 22943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:25.775620 22943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:25.777168 22943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "cf1fb48cc54d4452a192270b41125e72"
format_stamp: "Formatted at 2024-05-02 20:45:25 on dist-test-slave-g7g5"
I20240502 20:45:25.777598 22943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "cf1fb48cc54d4452a192270b41125e72"
format_stamp: "Formatted at 2024-05-02 20:45:25 on dist-test-slave-g7g5"
I20240502 20:45:25.780652 22943 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 20:45:25.782589 22957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:25.783495 22943 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:45:25.783602 22943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "cf1fb48cc54d4452a192270b41125e72"
format_stamp: "Formatted at 2024-05-02 20:45:25 on dist-test-slave-g7g5"
I20240502 20:45:25.783677 22943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:25.798601 22943 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:25.799165 22943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:25.799305 22943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:25.799782 22943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:25.800611 22943 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:25.800657 22943 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:25.800705 22943 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:25.800733 22943 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:25.808696 22943 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.66:40869
I20240502 20:45:25.808740 23070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.66:40869 every 8 connection(s)
I20240502 20:45:25.809530 22943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 20:45:25.815464 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 22943
I20240502 20:45:25.815587 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 20:45:25.817515 23071 heartbeater.cc:347] Connected to a master server at 127.22.45.126:46835
I20240502 20:45:25.817705 23071 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:25.817737 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.67:0
--local_ip_for_outbound_sockets=127.22.45.67
--webserver_interface=127.22.45.67
--webserver_port=0
--tserver_master_addrs=127.22.45.126:46835
--builtin_ntp_servers=127.22.45.84:39155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20240502 20:45:25.818218 23071 heartbeater.cc:511] Master 127.22.45.126:46835 requested a full tablet report, sending...
I20240502 20:45:25.819140 22752 ts_manager.cc:194] Registered new tserver with Master: cf1fb48cc54d4452a192270b41125e72 (127.22.45.66:40869)
I20240502 20:45:25.819665 22752 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.66:56095
W20240502 20:45:25.921041 23074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:25.921242 23074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:25.921286 23074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:25.924319 23074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:25.924443 23074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.67
I20240502 20:45:25.927975 23074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:39155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.22.45.67
--webserver_port=0
--tserver_master_addrs=127.22.45.126:46835
--never_fsync=true
--heap_profile_path=/tmp/kudu.23074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.67
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:25.928771 23074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:25.929811 23074 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:25.937325 23079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:25.937637 23080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:25.937990 23082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:25.938372 23074 server_base.cc:1034] running on GCE node
I20240502 20:45:25.938755 23074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:25.939304 23074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:25.940452 23074 hybrid_clock.cc:648] HybridClock initialized: now 1714682725940433 us; error 40 us; skew 500 ppm
I20240502 20:45:25.942324 23074 webserver.cc:460] Webserver started at http://127.22.45.67:42731/ using document root <none> and password file <none>
I20240502 20:45:25.942905 23074 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:25.942967 23074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:25.943154 23074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:25.944823 23074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "5c6363c7de194fc2bda2ec6d023417ae"
format_stamp: "Formatted at 2024-05-02 20:45:25 on dist-test-slave-g7g5"
I20240502 20:45:25.945303 23074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "5c6363c7de194fc2bda2ec6d023417ae"
format_stamp: "Formatted at 2024-05-02 20:45:25 on dist-test-slave-g7g5"
I20240502 20:45:25.948575 23074 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20240502 20:45:25.950640 23088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:25.951583 23074 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:45:25.951694 23074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "5c6363c7de194fc2bda2ec6d023417ae"
format_stamp: "Formatted at 2024-05-02 20:45:25 on dist-test-slave-g7g5"
I20240502 20:45:25.951771 23074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:25.980463 23074 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:25.981142 23074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:25.981299 23074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:25.981839 23074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:25.982759 23074 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:25.982807 23074 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:25.982860 23074 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:25.982883 23074 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:25.991422 23074 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.67:43341
I20240502 20:45:25.991452 23202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.67:43341 every 8 connection(s)
I20240502 20:45:25.992307 23074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 20:45:25.993239 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 23074
I20240502 20:45:25.993342 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 20:45:25.995388 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.68:0
--local_ip_for_outbound_sockets=127.22.45.68
--webserver_interface=127.22.45.68
--webserver_port=0
--tserver_master_addrs=127.22.45.126:46835
--builtin_ntp_servers=127.22.45.84:39155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20240502 20:45:26.001415 23203 heartbeater.cc:347] Connected to a master server at 127.22.45.126:46835
I20240502 20:45:26.001657 23203 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:26.002287 23203 heartbeater.cc:511] Master 127.22.45.126:46835 requested a full tablet report, sending...
I20240502 20:45:26.003383 22752 ts_manager.cc:194] Registered new tserver with Master: 5c6363c7de194fc2bda2ec6d023417ae (127.22.45.67:43341)
I20240502 20:45:26.003904 22752 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.67:35587
W20240502 20:45:26.101258 23206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:26.101461 23206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:26.101503 23206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:26.104472 23206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:26.104583 23206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.68
I20240502 20:45:26.108098 23206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:39155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.22.45.68
--webserver_port=0
--tserver_master_addrs=127.22.45.126:46835
--never_fsync=true
--heap_profile_path=/tmp/kudu.23206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.68
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:26.108916 23206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:26.109980 23206 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:26.117552 23214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:26.117868 23212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:26.118168 23211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:26.118625 23206 server_base.cc:1034] running on GCE node
I20240502 20:45:26.118997 23206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:26.119503 23206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:26.120644 23206 hybrid_clock.cc:648] HybridClock initialized: now 1714682726120620 us; error 38 us; skew 500 ppm
I20240502 20:45:26.122360 23206 webserver.cc:460] Webserver started at http://127.22.45.68:35453/ using document root <none> and password file <none>
I20240502 20:45:26.122889 23206 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:26.122951 23206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:26.123133 23206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:26.124747 23206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "b516ac252ea7420a8724773647dfc04d"
format_stamp: "Formatted at 2024-05-02 20:45:26 on dist-test-slave-g7g5"
I20240502 20:45:26.125185 23206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "b516ac252ea7420a8724773647dfc04d"
format_stamp: "Formatted at 2024-05-02 20:45:26 on dist-test-slave-g7g5"
I20240502 20:45:26.128397 23206 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 20:45:26.130448 23220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:26.131464 23206 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:45:26.131562 23206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "b516ac252ea7420a8724773647dfc04d"
format_stamp: "Formatted at 2024-05-02 20:45:26 on dist-test-slave-g7g5"
I20240502 20:45:26.131640 23206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:26.148196 23206 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:26.148806 23206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:26.148957 23206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:26.149461 23206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:26.150336 23206 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:26.150382 23206 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:26.150434 23206 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:26.150482 23206 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:26.158766 23206 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.68:38859
I20240502 20:45:26.158777 23334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.68:38859 every 8 connection(s)
I20240502 20:45:26.159672 23206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 20:45:26.160414 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 23206
I20240502 20:45:26.160511 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 20:45:26.162526 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.69:0
--local_ip_for_outbound_sockets=127.22.45.69
--webserver_interface=127.22.45.69
--webserver_port=0
--tserver_master_addrs=127.22.45.126:46835
--builtin_ntp_servers=127.22.45.84:39155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20240502 20:45:26.168515 23335 heartbeater.cc:347] Connected to a master server at 127.22.45.126:46835
I20240502 20:45:26.168720 23335 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:26.169209 23335 heartbeater.cc:511] Master 127.22.45.126:46835 requested a full tablet report, sending...
I20240502 20:45:26.170145 22752 ts_manager.cc:194] Registered new tserver with Master: b516ac252ea7420a8724773647dfc04d (127.22.45.68:38859)
I20240502 20:45:26.170707 22752 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.68:32781
W20240502 20:45:26.266137 23338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:26.266340 23338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:26.266395 23338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:26.269331 23338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:26.269452 23338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.69
I20240502 20:45:26.272933 23338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:39155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.69:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.22.45.69
--webserver_port=0
--tserver_master_addrs=127.22.45.126:46835
--never_fsync=true
--heap_profile_path=/tmp/kudu.23338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.69
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:26.273743 23338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:26.274839 23338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:26.282261 23344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:26.282584 23343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:26.282923 23346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:26.283305 23338 server_base.cc:1034] running on GCE node
I20240502 20:45:26.283605 23338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:26.284072 23338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:26.285210 23338 hybrid_clock.cc:648] HybridClock initialized: now 1714682726285202 us; error 22 us; skew 500 ppm
I20240502 20:45:26.286849 23338 webserver.cc:460] Webserver started at http://127.22.45.69:35427/ using document root <none> and password file <none>
I20240502 20:45:26.287362 23338 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:26.287415 23338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:26.287571 23338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:26.289249 23338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "b659e4a8ae3a4b42a66d9ff394c99977"
format_stamp: "Formatted at 2024-05-02 20:45:26 on dist-test-slave-g7g5"
I20240502 20:45:26.289712 23338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "b659e4a8ae3a4b42a66d9ff394c99977"
format_stamp: "Formatted at 2024-05-02 20:45:26 on dist-test-slave-g7g5"
I20240502 20:45:26.292824 23338 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20240502 20:45:26.294761 23352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:26.295662 23338 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:45:26.295753 23338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "b659e4a8ae3a4b42a66d9ff394c99977"
format_stamp: "Formatted at 2024-05-02 20:45:26 on dist-test-slave-g7g5"
I20240502 20:45:26.295833 23338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:26.309733 23338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:26.310367 23338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:26.310544 23338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:26.311065 23338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:26.311919 23338 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:26.311975 23338 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:26.312016 23338 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:26.312044 23338 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:26.320597 23338 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.69:33997
I20240502 20:45:26.320629 23466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.69:33997 every 8 connection(s)
I20240502 20:45:26.321453 23338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 20:45:26.328158 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 23338
I20240502 20:45:26.328279 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240502 20:45:26.329412 23467 heartbeater.cc:347] Connected to a master server at 127.22.45.126:46835
I20240502 20:45:26.329632 23467 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:26.330158 23467 heartbeater.cc:511] Master 127.22.45.126:46835 requested a full tablet report, sending...
I20240502 20:45:26.331123 22752 ts_manager.cc:194] Registered new tserver with Master: b659e4a8ae3a4b42a66d9ff394c99977 (127.22.45.69:33997)
I20240502 20:45:26.331643 22752 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.69:46231
I20240502 20:45:26.342399 22709 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240502 20:45:26.353080 22752 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53180:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240502 20:45:26.370261 23268 tablet_service.cc:1467] Processing CreateTablet for tablet 020aa8269314438d9b772e3c7ff0ddeb (DEFAULT_TABLE table=TestTable [id=6531c6f83c5147dd92f31c623c3d5030]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:26.370261 23136 tablet_service.cc:1467] Processing CreateTablet for tablet 020aa8269314438d9b772e3c7ff0ddeb (DEFAULT_TABLE table=TestTable [id=6531c6f83c5147dd92f31c623c3d5030]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:26.370205 22874 tablet_service.cc:1467] Processing CreateTablet for tablet 020aa8269314438d9b772e3c7ff0ddeb (DEFAULT_TABLE table=TestTable [id=6531c6f83c5147dd92f31c623c3d5030]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:26.371160 22874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 020aa8269314438d9b772e3c7ff0ddeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:26.371173 23136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 020aa8269314438d9b772e3c7ff0ddeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:26.371173 23268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 020aa8269314438d9b772e3c7ff0ddeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:26.375973 23484 tablet_bootstrap.cc:492] T 020aa8269314438d9b772e3c7ff0ddeb P 10c79541a38f410893c3b6474f9d0dae: Bootstrap starting.
I20240502 20:45:26.377238 23486 tablet_bootstrap.cc:492] T 020aa8269314438d9b772e3c7ff0ddeb P b516ac252ea7420a8724773647dfc04d: Bootstrap starting.
I20240502 20:45:26.377228 23485 tablet_bootstrap.cc:492] T 020aa8269314438d9b772e3c7ff0ddeb P 5c6363c7de194fc2bda2ec6d023417ae: Bootstrap starting.
I20240502 20:45:26.378015 23484 tablet_bootstrap.cc:654] T 020aa8269314438d9b772e3c7ff0ddeb P 10c79541a38f410893c3b6474f9d0dae: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:26.379050 23484 log.cc:824] T 020aa8269314438d9b772e3c7ff0ddeb P 10c79541a38f410893c3b6474f9d0dae: Log is configured to *not* fsync() on all Append() calls
I20240502 20:45:26.379227 23486 tablet_bootstrap.cc:654] T 020aa8269314438d9b772e3c7ff0ddeb P b516ac252ea7420a8724773647dfc04d: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:26.379227 23485 tablet_bootstrap.cc:654] T 020aa8269314438d9b772e3c7ff0ddeb P 5c6363c7de194fc2bda2ec6d023417ae: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:26.379978 23486 log.cc:824] T 020aa8269314438d9b772e3c7ff0ddeb P b516ac252ea7420a8724773647dfc04d: Log is configured to *not* fsync() on all Append() calls
I20240502 20:45:26.379969 23485 log.cc:824] T 020aa8269314438d9b772e3c7ff0ddeb P 5c6363c7de194fc2bda2ec6d023417ae: Log is configured to *not* fsync() on all Append() calls
I20240502 20:45:26.380564 23484 tablet_bootstrap.cc:492] T 020aa8269314438d9b772e3c7ff0ddeb P 10c79541a38f410893c3b6474f9d0dae: No bootstrap required, opened a new log
I20240502 20:45:26.380743 23484 ts_tablet_manager.cc:1367] T 020aa8269314438d9b772e3c7ff0ddeb P 10c79541a38f410893c3b6474f9d0dae: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240502 20:45:26.381496 23486 tablet_bootstrap.cc:492] T 020aa8269314438d9b772e3c7ff0ddeb P b516ac252ea7420a8724773647dfc04d: No bootstrap required, opened a new log
I20240502 20:45:26.381665 23486 ts_tablet_manager.cc:1367] T 020aa8269314438d9b772e3c7ff0ddeb P b516ac252ea7420a8724773647dfc04d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240502 20:45:26.381783 23485 tablet_bootstrap.cc:492] T 020aa8269314438d9b772e3c7ff0ddeb P 5c6363c7de194fc2bda2ec6d023417ae: No bootstrap required, opened a new log
I20240502 20:45:26.381939 23485 ts_tablet_manager.cc:1367] T 020aa8269314438d9b772e3c7ff0ddeb P 5c6363c7de194fc2bda2ec6d023417ae: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240502 20:45:26.383291 23484 raft_consensus.cc:348] T 020aa8269314438d9b772e3c7ff0ddeb P 10c79541a38f410893c3b6474f9d0dae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c6363c7de194fc2bda2ec6d023417ae" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 43341 } } peers { permanent_uuid: "10c79541a38f410893c3b6474f9d0dae" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 33845 } } peers { permanent_uuid: "b516ac252ea7420a8724773647dfc04d" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 38859 } }
I20240502 20:45:26.383479 23484 raft_consensus.cc:374] T 020aa8269314438d9b772e3c7ff0ddeb P 10c79541a38f410893c3b6474f9d0dae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:26.383551 23484 raft_consensus.cc:729] T 020aa8269314438d9b772e3c7ff0ddeb P 10c79541a38f410893c3b6474f9d0dae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10c79541a38f410893c3b6474f9d0dae, State: Initialized, Role: FOLLOWER
I20240502 20:45:26.384127 23484 consensus_queue.cc:260] T 020aa8269314438d9b772e3c7ff0ddeb P 10c79541a38f410893c3b6474f9d0dae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c6363c7de194fc2bda2ec6d023417ae" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 43341 } } peers { permanent_uuid: "10c79541a38f410893c3b6474f9d0dae" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 33845 } } peers { permanent_uuid: "b516ac252ea7420a8724773647dfc04d" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 38859 } }
I20240502 20:45:26.384922 23484 ts_tablet_manager.cc:1398] T 020aa8269314438d9b772e3c7ff0ddeb P 10c79541a38f410893c3b6474f9d0dae: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20240502 20:45:26.385257 22940 heartbeater.cc:503] Master 127.22.45.126:46835 was elected leader, sending a full tablet report...
I20240502 20:45:26.385274 23486 raft_consensus.cc:348] T 020aa8269314438d9b772e3c7ff0ddeb P b516ac252ea7420a8724773647dfc04d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c6363c7de194fc2bda2ec6d023417ae" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 43341 } } peers { permanent_uuid: "10c79541a38f410893c3b6474f9d0dae" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 33845 } } peers { permanent_uuid: "b516ac252ea7420a8724773647dfc04d" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 38859 } }
I20240502 20:45:26.385458 23486 raft_consensus.cc:374] T 020aa8269314438d9b772e3c7ff0ddeb P b516ac252ea7420a8724773647dfc04d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:26.385522 23486 raft_consensus.cc:729] T 020aa8269314438d9b772e3c7ff0ddeb P b516ac252ea7420a8724773647dfc04d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b516ac252ea7420a8724773647dfc04d, State: Initialized, Role: FOLLOWER
I20240502 20:45:26.385946 23486 consensus_queue.cc:260] T 020aa8269314438d9b772e3c7ff0ddeb P b516ac252ea7420a8724773647dfc04d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c6363c7de194fc2bda2ec6d023417ae" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 43341 } } peers { permanent_uuid: "10c79541a38f410893c3b6474f9d0dae" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 33845 } } peers { permanent_uuid: "b516ac252ea7420a8724773647dfc04d" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 38859 } }
I20240502 20:45:26.386641 23335 heartbeater.cc:503] Master 127.22.45.126:46835 was elected leader, sending a full tablet report...
I20240502 20:45:26.386680 23485 raft_consensus.cc:348] T 020aa8269314438d9b772e3c7ff0ddeb P 5c6363c7de194fc2bda2ec6d023417ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c6363c7de194fc2bda2ec6d023417ae" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 43341 } } peers { permanent_uuid: "10c79541a38f410893c3b6474f9d0dae" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 33845 } } peers { permanent_uuid: "b516ac252ea7420a8724773647dfc04d" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 38859 } }
I20240502 20:45:26.386842 23486 ts_tablet_manager.cc:1398] T 020aa8269314438d9b772e3c7ff0ddeb P b516ac252ea7420a8724773647dfc04d: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20240502 20:45:26.386916 23485 raft_consensus.cc:374] T 020aa8269314438d9b772e3c7ff0ddeb P 5c6363c7de194fc2bda2ec6d023417ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:26.386997 23485 raft_consensus.cc:729] T 020aa8269314438d9b772e3c7ff0ddeb P 5c6363c7de194fc2bda2ec6d023417ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c6363c7de194fc2bda2ec6d023417ae, State: Initialized, Role: FOLLOWER
I20240502 20:45:26.387426 23485 consensus_queue.cc:260] T 020aa8269314438d9b772e3c7ff0ddeb P 5c6363c7de194fc2bda2ec6d023417ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c6363c7de194fc2bda2ec6d023417ae" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 43341 } } peers { permanent_uuid: "10c79541a38f410893c3b6474f9d0dae" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 33845 } } peers { permanent_uuid: "b516ac252ea7420a8724773647dfc04d" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 38859 } }
I20240502 20:45:26.388111 23485 ts_tablet_manager.cc:1398] T 020aa8269314438d9b772e3c7ff0ddeb P 5c6363c7de194fc2bda2ec6d023417ae: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240502 20:45:26.388454 23203 heartbeater.cc:503] Master 127.22.45.126:46835 was elected leader, sending a full tablet report...
I20240502 20:45:26.394057 23491 raft_consensus.cc:482] T 020aa8269314438d9b772e3c7ff0ddeb P b516ac252ea7420a8724773647dfc04d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:45:26.394174 23491 raft_consensus.cc:504] T 020aa8269314438d9b772e3c7ff0ddeb P b516ac252ea7420a8724773647dfc04d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c6363c7de194fc2bda2ec6d023417ae" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 43341 } } peers { permanent_uuid: "10c79541a38f410893c3b6474f9d0dae" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 33845 } } peers { permanent_uuid: "b516ac252ea7420a8724773647dfc04d" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 38859 } }
I20240502 20:45:26.395036 23491 leader_election.cc:288] T 020aa8269314438d9b772e3c7ff0ddeb P b516ac252ea7420a8724773647dfc04d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c6363c7de194fc2bda2ec6d023417ae (127.22.45.67:43341), 10c79541a38f410893c3b6474f9d0dae (127.22.45.65:33845)
W20240502 20:45:26.397802 22941 tablet.cc:2369] T 020aa8269314438d9b772e3c7ff0ddeb P 10c79541a38f410893c3b6474f9d0dae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 20:45:26.398631 23156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "020aa8269314438d9b772e3c7ff0ddeb" candidate_uuid: "b516ac252ea7420a8724773647dfc04d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c6363c7de194fc2bda2ec6d023417ae" is_pre_election: true
I20240502 20:45:26.398883 23156 raft_consensus.cc:2454] T 020aa8269314438d9b772e3c7ff0ddeb P 5c6363c7de194fc2bda2ec6d023417ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b516ac252ea7420a8724773647dfc04d in term 0.
I20240502 20:45:26.399082 22894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "020aa8269314438d9b772e3c7ff0ddeb" candidate_uuid: "b516ac252ea7420a8724773647dfc04d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10c79541a38f410893c3b6474f9d0dae" is_pre_election: true
I20240502 20:45:26.399310 23223 leader_election.cc:302] T 020aa8269314438d9b772e3c7ff0ddeb P b516ac252ea7420a8724773647dfc04d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c6363c7de194fc2bda2ec6d023417ae, b516ac252ea7420a8724773647dfc04d; no voters:
I20240502 20:45:26.399339 22894 raft_consensus.cc:2454] T 020aa8269314438d9b772e3c7ff0ddeb P 10c79541a38f410893c3b6474f9d0dae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b516ac252ea7420a8724773647dfc04d in term 0.
I20240502 20:45:26.399523 23491 raft_consensus.cc:2781] T 020aa8269314438d9b772e3c7ff0ddeb P b516ac252ea7420a8724773647dfc04d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:45:26.399600 23491 raft_consensus.cc:482] T 020aa8269314438d9b772e3c7ff0ddeb P b516ac252ea7420a8724773647dfc04d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:45:26.399641 23491 raft_consensus.cc:3037] T 020aa8269314438d9b772e3c7ff0ddeb P b516ac252ea7420a8724773647dfc04d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:26.400528 23491 raft_consensus.cc:504] T 020aa8269314438d9b772e3c7ff0ddeb P b516ac252ea7420a8724773647dfc04d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c6363c7de194fc2bda2ec6d023417ae" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 43341 } } peers { permanent_uuid: "10c79541a38f410893c3b6474f9d0dae" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 33845 } } peers { permanent_uuid: "b516ac252ea7420a8724773647dfc04d" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 38859 } }
I20240502 20:45:26.400804 23491 leader_election.cc:288] T 020aa8269314438d9b772e3c7ff0ddeb P b516ac252ea7420a8724773647dfc04d [CANDIDATE]: Term 1 election: Requested vote from peers 5c6363c7de194fc2bda2ec6d023417ae (127.22.45.67:43341), 10c79541a38f410893c3b6474f9d0dae (127.22.45.65:33845)
I20240502 20:45:26.401079 23156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "020aa8269314438d9b772e3c7ff0ddeb" candidate_uuid: "b516ac252ea7420a8724773647dfc04d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c6363c7de194fc2bda2ec6d023417ae"
I20240502 20:45:26.401187 23156 raft_consensus.cc:3037] T 020aa8269314438d9b772e3c7ff0ddeb P 5c6363c7de194fc2bda2ec6d023417ae [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:26.401178 22894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "020aa8269314438d9b772e3c7ff0ddeb" candidate_uuid: "b516ac252ea7420a8724773647dfc04d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10c79541a38f410893c3b6474f9d0dae"
I20240502 20:45:26.401280 22894 raft_consensus.cc:3037] T 020aa8269314438d9b772e3c7ff0ddeb P 10c79541a38f410893c3b6474f9d0dae [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:26.401979 23156 raft_consensus.cc:2454] T 020aa8269314438d9b772e3c7ff0ddeb P 5c6363c7de194fc2bda2ec6d023417ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b516ac252ea7420a8724773647dfc04d in term 1.
I20240502 20:45:26.402069 22894 raft_consensus.cc:2454] T 020aa8269314438d9b772e3c7ff0ddeb P 10c79541a38f410893c3b6474f9d0dae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b516ac252ea7420a8724773647dfc04d in term 1.
I20240502 20:45:26.402248 23223 leader_election.cc:302] T 020aa8269314438d9b772e3c7ff0ddeb P b516ac252ea7420a8724773647dfc04d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c6363c7de194fc2bda2ec6d023417ae, b516ac252ea7420a8724773647dfc04d; no voters:
I20240502 20:45:26.402407 23491 raft_consensus.cc:2781] T 020aa8269314438d9b772e3c7ff0ddeb P b516ac252ea7420a8724773647dfc04d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:45:26.402616 23491 raft_consensus.cc:686] T 020aa8269314438d9b772e3c7ff0ddeb P b516ac252ea7420a8724773647dfc04d [term 1 LEADER]: Becoming Leader. State: Replica: b516ac252ea7420a8724773647dfc04d, State: Running, Role: LEADER
I20240502 20:45:26.402880 23491 consensus_queue.cc:237] T 020aa8269314438d9b772e3c7ff0ddeb P b516ac252ea7420a8724773647dfc04d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c6363c7de194fc2bda2ec6d023417ae" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 43341 } } peers { permanent_uuid: "10c79541a38f410893c3b6474f9d0dae" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 33845 } } peers { permanent_uuid: "b516ac252ea7420a8724773647dfc04d" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 38859 } }
I20240502 20:45:26.405347 22752 catalog_manager.cc:5484] T 020aa8269314438d9b772e3c7ff0ddeb P b516ac252ea7420a8724773647dfc04d reported cstate change: term changed from 0 to 1, leader changed from <none> to b516ac252ea7420a8724773647dfc04d (127.22.45.68). New cstate: current_term: 1 leader_uuid: "b516ac252ea7420a8724773647dfc04d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c6363c7de194fc2bda2ec6d023417ae" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 43341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10c79541a38f410893c3b6474f9d0dae" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 33845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b516ac252ea7420a8724773647dfc04d" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 38859 } health_report { overall_health: HEALTHY } } }
W20240502 20:45:26.410581 23336 tablet.cc:2369] T 020aa8269314438d9b772e3c7ff0ddeb P b516ac252ea7420a8724773647dfc04d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 20:45:26.414294 22709 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240502 20:45:26.415782 22709 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 10c79541a38f410893c3b6474f9d0dae to finish bootstrapping
I20240502 20:45:26.422271 22709 ts_itest-base.cc:233] Waiting for 0 tablets on tserver cf1fb48cc54d4452a192270b41125e72 to finish bootstrapping
I20240502 20:45:26.427556 22709 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 5c6363c7de194fc2bda2ec6d023417ae to finish bootstrapping
I20240502 20:45:26.431717 22709 ts_itest-base.cc:233] Waiting for 1 tablets on tserver b516ac252ea7420a8724773647dfc04d to finish bootstrapping
I20240502 20:45:26.435632 22709 ts_itest-base.cc:233] Waiting for 0 tablets on tserver b659e4a8ae3a4b42a66d9ff394c99977 to finish bootstrapping
W20240502 20:45:26.493348 23204 tablet.cc:2369] T 020aa8269314438d9b772e3c7ff0ddeb P 5c6363c7de194fc2bda2ec6d023417ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 20:45:26.612881 23136 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:26.612881 22874 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:26.614199 23400 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:26.623726 23268 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:26.623792 23005 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:26.702656 23005 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:26.702714 23136 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:26.702966 22874 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:26.704216 23400 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:26.705472 23268 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:26.742014 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 22812
I20240502 20:45:26.750046 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 22943
I20240502 20:45:26.756570 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 23074
I20240502 20:45:26.761029 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 23206
I20240502 20:45:26.765578 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 23338
I20240502 20:45:26.770315 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 22721
2024-05-02T20:45:26Z chronyd exiting
[ OK ] AdminCliTest.RebalancerReportOnly (1556 ms)
[----------] 1 test from AdminCliTest (1556 ms total)
[----------] 1 test from RebalanceStartCriteriaTest
[ RUN ] RebalanceStartCriteriaTest.TabletServerIsDown/0
I20240502 20:45:26.790663 22709 test_util.cc:268] Using random seed: -1361758718
I20240502 20:45:26.791340 22709 ts_itest-base.cc:112] Starting cluster with:
I20240502 20:45:26.791380 22709 ts_itest-base.cc:113] --------------
I20240502 20:45:26.791395 22709 ts_itest-base.cc:114] 5 tablet servers
I20240502 20:45:26.791409 22709 ts_itest-base.cc:115] 3 replicas per TS
I20240502 20:45:26.791421 22709 ts_itest-base.cc:116] --------------
2024-05-02T20:45:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T20:45:26Z Disabled control of system clock
I20240502 20:45:26.798621 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.45.126:35441
--webserver_interface=127.22.45.126
--webserver_port=0
--builtin_ntp_servers=127.22.45.84:37221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.45.126:35441
--raft_prepare_replacement_before_eviction=false
--enable_range_replica_placement=false with env {}
W20240502 20:45:26.898248 23573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:26.898449 23573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:26.898501 23573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:26.901325 23573 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:45:26.901379 23573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 20:45:26.901400 23573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:26.901425 23573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 20:45:26.901440 23573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 20:45:26.904911 23573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:37221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.22.45.126:35441
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.45.126:35441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.22.45.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.23573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:26.905652 23573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:26.906662 23573 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:26.913376 23579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:26.913617 23581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:26.913957 23578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:26.914382 23573 server_base.cc:1034] running on GCE node
I20240502 20:45:26.914827 23573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:26.915638 23573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:26.916780 23573 hybrid_clock.cc:648] HybridClock initialized: now 1714682726916772 us; error 25 us; skew 500 ppm
I20240502 20:45:26.918545 23573 webserver.cc:460] Webserver started at http://127.22.45.126:35675/ using document root <none> and password file <none>
I20240502 20:45:26.919039 23573 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:26.919095 23573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:26.919255 23573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:26.920779 23573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "aba7005c3a244fe8b6a1a29aca813ee6"
format_stamp: "Formatted at 2024-05-02 20:45:26 on dist-test-slave-g7g5"
I20240502 20:45:26.921205 23573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "aba7005c3a244fe8b6a1a29aca813ee6"
format_stamp: "Formatted at 2024-05-02 20:45:26 on dist-test-slave-g7g5"
I20240502 20:45:26.924263 23573 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 20:45:26.926178 23587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:26.927112 23573 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:45:26.927222 23573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal
uuid: "aba7005c3a244fe8b6a1a29aca813ee6"
format_stamp: "Formatted at 2024-05-02 20:45:26 on dist-test-slave-g7g5"
I20240502 20:45:26.927295 23573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:26.946128 23573 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:26.946712 23573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:26.946854 23573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:26.952843 23573 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.126:35441
I20240502 20:45:26.952848 23639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.126:35441 every 8 connection(s)
I20240502 20:45:26.953730 23573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 20:45:26.956099 23640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:26.960772 23640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aba7005c3a244fe8b6a1a29aca813ee6: Bootstrap starting.
I20240502 20:45:26.962869 23640 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aba7005c3a244fe8b6a1a29aca813ee6: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:26.962888 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 23573
I20240502 20:45:26.963001 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 20:45:26.963531 23640 log.cc:824] T 00000000000000000000000000000000 P aba7005c3a244fe8b6a1a29aca813ee6: Log is configured to *not* fsync() on all Append() calls
I20240502 20:45:26.965397 23640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aba7005c3a244fe8b6a1a29aca813ee6: No bootstrap required, opened a new log
I20240502 20:45:26.968425 23640 raft_consensus.cc:348] T 00000000000000000000000000000000 P aba7005c3a244fe8b6a1a29aca813ee6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba7005c3a244fe8b6a1a29aca813ee6" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 35441 } }
I20240502 20:45:26.968633 23640 raft_consensus.cc:374] T 00000000000000000000000000000000 P aba7005c3a244fe8b6a1a29aca813ee6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:26.968699 23640 raft_consensus.cc:729] T 00000000000000000000000000000000 P aba7005c3a244fe8b6a1a29aca813ee6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aba7005c3a244fe8b6a1a29aca813ee6, State: Initialized, Role: FOLLOWER
I20240502 20:45:26.969141 23640 consensus_queue.cc:260] T 00000000000000000000000000000000 P aba7005c3a244fe8b6a1a29aca813ee6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba7005c3a244fe8b6a1a29aca813ee6" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 35441 } }
I20240502 20:45:26.969282 23640 raft_consensus.cc:388] T 00000000000000000000000000000000 P aba7005c3a244fe8b6a1a29aca813ee6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 20:45:26.969347 23640 raft_consensus.cc:482] T 00000000000000000000000000000000 P aba7005c3a244fe8b6a1a29aca813ee6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 20:45:26.969458 23640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aba7005c3a244fe8b6a1a29aca813ee6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:26.970322 23640 raft_consensus.cc:504] T 00000000000000000000000000000000 P aba7005c3a244fe8b6a1a29aca813ee6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba7005c3a244fe8b6a1a29aca813ee6" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 35441 } }
I20240502 20:45:26.970659 23640 leader_election.cc:302] T 00000000000000000000000000000000 P aba7005c3a244fe8b6a1a29aca813ee6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aba7005c3a244fe8b6a1a29aca813ee6; no voters:
I20240502 20:45:26.970886 23640 leader_election.cc:288] T 00000000000000000000000000000000 P aba7005c3a244fe8b6a1a29aca813ee6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 20:45:26.970984 23645 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aba7005c3a244fe8b6a1a29aca813ee6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:45:26.971187 23645 raft_consensus.cc:686] T 00000000000000000000000000000000 P aba7005c3a244fe8b6a1a29aca813ee6 [term 1 LEADER]: Becoming Leader. State: Replica: aba7005c3a244fe8b6a1a29aca813ee6, State: Running, Role: LEADER
I20240502 20:45:26.971453 23645 consensus_queue.cc:237] T 00000000000000000000000000000000 P aba7005c3a244fe8b6a1a29aca813ee6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba7005c3a244fe8b6a1a29aca813ee6" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 35441 } }
I20240502 20:45:26.971923 23640 sys_catalog.cc:564] T 00000000000000000000000000000000 P aba7005c3a244fe8b6a1a29aca813ee6 [sys.catalog]: configured and running, proceeding with master startup.
I20240502 20:45:26.973004 23647 sys_catalog.cc:455] T 00000000000000000000000000000000 P aba7005c3a244fe8b6a1a29aca813ee6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aba7005c3a244fe8b6a1a29aca813ee6. Latest consensus state: current_term: 1 leader_uuid: "aba7005c3a244fe8b6a1a29aca813ee6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba7005c3a244fe8b6a1a29aca813ee6" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 35441 } } }
I20240502 20:45:26.973127 23647 sys_catalog.cc:458] T 00000000000000000000000000000000 P aba7005c3a244fe8b6a1a29aca813ee6 [sys.catalog]: This master's current role is: LEADER
I20240502 20:45:26.973213 23646 sys_catalog.cc:455] T 00000000000000000000000000000000 P aba7005c3a244fe8b6a1a29aca813ee6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aba7005c3a244fe8b6a1a29aca813ee6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba7005c3a244fe8b6a1a29aca813ee6" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 35441 } } }
I20240502 20:45:26.973285 23646 sys_catalog.cc:458] T 00000000000000000000000000000000 P aba7005c3a244fe8b6a1a29aca813ee6 [sys.catalog]: This master's current role is: LEADER
I20240502 20:45:26.973486 23649 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 20:45:26.976189 23649 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 20:45:26.981009 23649 catalog_manager.cc:1347] Generated new cluster ID: ef03f7374c5746cbad36d5b7291a1470
I20240502 20:45:26.981072 23649 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 20:45:26.989665 23649 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 20:45:26.990883 23649 catalog_manager.cc:1504] Loading token signing keys...
I20240502 20:45:26.999769 23649 catalog_manager.cc:5857] T 00000000000000000000000000000000 P aba7005c3a244fe8b6a1a29aca813ee6: Generated new TSK 0
I20240502 20:45:27.000409 23649 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 20:45:27.011845 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.65:0
--local_ip_for_outbound_sockets=127.22.45.65
--webserver_interface=127.22.45.65
--webserver_port=0
--tserver_master_addrs=127.22.45.126:35441
--builtin_ntp_servers=127.22.45.84:37221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 20:45:27.133612 23664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:27.133803 23664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:27.133846 23664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:27.136794 23664 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:45:27.136847 23664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:27.136919 23664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.65
I20240502 20:45:27.140712 23664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:37221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.22.45.65
--webserver_port=0
--tserver_master_addrs=127.22.45.126:35441
--never_fsync=true
--heap_profile_path=/tmp/kudu.23664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.65
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:27.141507 23664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:27.142583 23664 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:27.150628 23672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:27.150839 23669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:27.151160 23670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:27.151557 23664 server_base.cc:1034] running on GCE node
I20240502 20:45:27.151911 23664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:27.152427 23664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:27.153568 23664 hybrid_clock.cc:648] HybridClock initialized: now 1714682727153555 us; error 29 us; skew 500 ppm
I20240502 20:45:27.155278 23664 webserver.cc:460] Webserver started at http://127.22.45.65:37759/ using document root <none> and password file <none>
I20240502 20:45:27.155773 23664 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:27.155829 23664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:27.155997 23664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:27.157549 23664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "57bb4cc0be2f4b6e94c119f7a39d7acf"
format_stamp: "Formatted at 2024-05-02 20:45:27 on dist-test-slave-g7g5"
I20240502 20:45:27.157990 23664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "57bb4cc0be2f4b6e94c119f7a39d7acf"
format_stamp: "Formatted at 2024-05-02 20:45:27 on dist-test-slave-g7g5"
I20240502 20:45:27.160972 23664 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 20:45:27.162920 23678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:27.163810 23664 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:45:27.163906 23664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "57bb4cc0be2f4b6e94c119f7a39d7acf"
format_stamp: "Formatted at 2024-05-02 20:45:27 on dist-test-slave-g7g5"
I20240502 20:45:27.163990 23664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:27.180953 23664 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:27.181461 23664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:27.181602 23664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:27.182062 23664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:27.182921 23664 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:27.182973 23664 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:27.183023 23664 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:27.183051 23664 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:27.190690 23664 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.65:40707
I20240502 20:45:27.190747 23792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.65:40707 every 8 connection(s)
I20240502 20:45:27.191550 23664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 20:45:27.197609 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 23664
I20240502 20:45:27.197715 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 20:45:27.199788 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.66:0
--local_ip_for_outbound_sockets=127.22.45.66
--webserver_interface=127.22.45.66
--webserver_port=0
--tserver_master_addrs=127.22.45.126:35441
--builtin_ntp_servers=127.22.45.84:37221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:45:27.199980 23794 heartbeater.cc:347] Connected to a master server at 127.22.45.126:35441
I20240502 20:45:27.200171 23794 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:27.200556 23794 heartbeater.cc:511] Master 127.22.45.126:35441 requested a full tablet report, sending...
I20240502 20:45:27.201815 23604 ts_manager.cc:194] Registered new tserver with Master: 57bb4cc0be2f4b6e94c119f7a39d7acf (127.22.45.65:40707)
I20240502 20:45:27.203336 23604 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.65:50921
W20240502 20:45:27.299690 23797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:27.299885 23797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:27.299925 23797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:27.302776 23797 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:45:27.302827 23797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:27.302887 23797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.66
I20240502 20:45:27.306236 23797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:37221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.22.45.66
--webserver_port=0
--tserver_master_addrs=127.22.45.126:35441
--never_fsync=true
--heap_profile_path=/tmp/kudu.23797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.66
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:27.306977 23797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:27.307962 23797 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:27.315428 23805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:27.315675 23802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:27.316020 23803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:27.316413 23797 server_base.cc:1034] running on GCE node
I20240502 20:45:27.316751 23797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:27.317248 23797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:27.318385 23797 hybrid_clock.cc:648] HybridClock initialized: now 1714682727318373 us; error 26 us; skew 500 ppm
I20240502 20:45:27.320057 23797 webserver.cc:460] Webserver started at http://127.22.45.66:36521/ using document root <none> and password file <none>
I20240502 20:45:27.320540 23797 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:27.320595 23797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:27.320749 23797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:27.322252 23797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "01ea59220a9b47e0ab797f58191bdb72"
format_stamp: "Formatted at 2024-05-02 20:45:27 on dist-test-slave-g7g5"
I20240502 20:45:27.322696 23797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "01ea59220a9b47e0ab797f58191bdb72"
format_stamp: "Formatted at 2024-05-02 20:45:27 on dist-test-slave-g7g5"
I20240502 20:45:27.325578 23797 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20240502 20:45:27.327500 23811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:27.328367 23797 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 20:45:27.328462 23797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "01ea59220a9b47e0ab797f58191bdb72"
format_stamp: "Formatted at 2024-05-02 20:45:27 on dist-test-slave-g7g5"
I20240502 20:45:27.328537 23797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:27.343919 23797 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:27.344391 23797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:27.344527 23797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:27.344969 23797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:27.345742 23797 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:27.345788 23797 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:27.345834 23797 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:27.345862 23797 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:27.353318 23797 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.66:46801
I20240502 20:45:27.353363 23925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.66:46801 every 8 connection(s)
I20240502 20:45:27.354121 23797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 20:45:27.354854 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 23797
I20240502 20:45:27.354960 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 20:45:27.357019 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.67:0
--local_ip_for_outbound_sockets=127.22.45.67
--webserver_interface=127.22.45.67
--webserver_port=0
--tserver_master_addrs=127.22.45.126:35441
--builtin_ntp_servers=127.22.45.84:37221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:45:27.362843 23926 heartbeater.cc:347] Connected to a master server at 127.22.45.126:35441
I20240502 20:45:27.363037 23926 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:27.363421 23926 heartbeater.cc:511] Master 127.22.45.126:35441 requested a full tablet report, sending...
I20240502 20:45:27.364288 23604 ts_manager.cc:194] Registered new tserver with Master: 01ea59220a9b47e0ab797f58191bdb72 (127.22.45.66:46801)
I20240502 20:45:27.364799 23604 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.66:40591
W20240502 20:45:27.457422 23929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:27.457612 23929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:27.457667 23929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:27.460603 23929 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:45:27.460667 23929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:27.460745 23929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.67
I20240502 20:45:27.464182 23929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:37221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.22.45.67
--webserver_port=0
--tserver_master_addrs=127.22.45.126:35441
--never_fsync=true
--heap_profile_path=/tmp/kudu.23929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.67
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:27.464979 23929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:27.465999 23929 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:27.473682 23934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:27.473968 23935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:27.474216 23937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:27.474637 23929 server_base.cc:1034] running on GCE node
I20240502 20:45:27.474941 23929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:27.475407 23929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:27.476560 23929 hybrid_clock.cc:648] HybridClock initialized: now 1714682727476537 us; error 35 us; skew 500 ppm
I20240502 20:45:27.478214 23929 webserver.cc:460] Webserver started at http://127.22.45.67:43971/ using document root <none> and password file <none>
I20240502 20:45:27.478739 23929 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:27.478793 23929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:27.478966 23929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:27.480484 23929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "2ecf233910ac499d91f656a4d9940dcf"
format_stamp: "Formatted at 2024-05-02 20:45:27 on dist-test-slave-g7g5"
I20240502 20:45:27.480916 23929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "2ecf233910ac499d91f656a4d9940dcf"
format_stamp: "Formatted at 2024-05-02 20:45:27 on dist-test-slave-g7g5"
I20240502 20:45:27.484045 23929 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20240502 20:45:27.485978 23943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:27.486904 23929 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:45:27.487010 23929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "2ecf233910ac499d91f656a4d9940dcf"
format_stamp: "Formatted at 2024-05-02 20:45:27 on dist-test-slave-g7g5"
I20240502 20:45:27.487084 23929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:27.503172 23929 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:27.503669 23929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:27.503808 23929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:27.504273 23929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:27.505065 23929 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:27.505111 23929 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:27.505160 23929 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:27.505187 23929 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:27.513005 23929 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.67:38893
I20240502 20:45:27.513135 24057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.67:38893 every 8 connection(s)
I20240502 20:45:27.517692 23929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 20:45:27.522150 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 23929
I20240502 20:45:27.522230 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 20:45:27.524109 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.68:0
--local_ip_for_outbound_sockets=127.22.45.68
--webserver_interface=127.22.45.68
--webserver_port=0
--tserver_master_addrs=127.22.45.126:35441
--builtin_ntp_servers=127.22.45.84:37221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:45:27.530522 24058 heartbeater.cc:347] Connected to a master server at 127.22.45.126:35441
I20240502 20:45:27.530732 24058 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:27.531253 24058 heartbeater.cc:511] Master 127.22.45.126:35441 requested a full tablet report, sending...
I20240502 20:45:27.532332 23604 ts_manager.cc:194] Registered new tserver with Master: 2ecf233910ac499d91f656a4d9940dcf (127.22.45.67:38893)
I20240502 20:45:27.533072 23604 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.67:44217
W20240502 20:45:27.626328 24061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:27.626636 24061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:27.626727 24061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:27.630054 24061 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:45:27.630111 24061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:27.630203 24061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.68
I20240502 20:45:27.633822 24061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:37221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.22.45.68
--webserver_port=0
--tserver_master_addrs=127.22.45.126:35441
--never_fsync=true
--heap_profile_path=/tmp/kudu.24061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.68
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:27.634642 24061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:27.635663 24061 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:27.644083 24069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:27.644259 24066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:27.644651 24067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:27.645215 24061 server_base.cc:1034] running on GCE node
I20240502 20:45:27.645624 24061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:27.646251 24061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:27.647425 24061 hybrid_clock.cc:648] HybridClock initialized: now 1714682727647401 us; error 40 us; skew 500 ppm
I20240502 20:45:27.649621 24061 webserver.cc:460] Webserver started at http://127.22.45.68:35869/ using document root <none> and password file <none>
I20240502 20:45:27.650276 24061 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:27.650348 24061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:27.650576 24061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:27.652685 24061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "70053cc6a9b74bf18b71345465585c60"
format_stamp: "Formatted at 2024-05-02 20:45:27 on dist-test-slave-g7g5"
I20240502 20:45:27.653249 24061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "70053cc6a9b74bf18b71345465585c60"
format_stamp: "Formatted at 2024-05-02 20:45:27 on dist-test-slave-g7g5"
I20240502 20:45:27.659049 24061 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20240502 20:45:27.665077 24075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:27.666278 24061 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20240502 20:45:27.666396 24061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "70053cc6a9b74bf18b71345465585c60"
format_stamp: "Formatted at 2024-05-02 20:45:27 on dist-test-slave-g7g5"
I20240502 20:45:27.666508 24061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:27.689663 24061 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:27.690505 24061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:27.690671 24061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:27.691392 24061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:27.692414 24061 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:27.692464 24061 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:27.692516 24061 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:27.692546 24061 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:27.702448 24061 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.68:35935
I20240502 20:45:27.702500 24189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.68:35935 every 8 connection(s)
I20240502 20:45:27.703593 24061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 20:45:27.710222 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 24061
I20240502 20:45:27.710330 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 20:45:27.712586 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.69:0
--local_ip_for_outbound_sockets=127.22.45.69
--webserver_interface=127.22.45.69
--webserver_port=0
--tserver_master_addrs=127.22.45.126:35441
--builtin_ntp_servers=127.22.45.84:37221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:45:27.714272 24190 heartbeater.cc:347] Connected to a master server at 127.22.45.126:35441
I20240502 20:45:27.714538 24190 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:27.715057 24190 heartbeater.cc:511] Master 127.22.45.126:35441 requested a full tablet report, sending...
I20240502 20:45:27.716042 23604 ts_manager.cc:194] Registered new tserver with Master: 70053cc6a9b74bf18b71345465585c60 (127.22.45.68:35935)
I20240502 20:45:27.716780 23604 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.68:34507
W20240502 20:45:27.814571 24193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:27.814759 24193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:27.814817 24193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:27.817728 24193 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:45:27.817785 24193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:27.817865 24193 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.69
I20240502 20:45:27.821251 24193 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:37221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.69:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.22.45.69
--webserver_port=0
--tserver_master_addrs=127.22.45.126:35441
--never_fsync=true
--heap_profile_path=/tmp/kudu.24193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.69
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:27.822016 24193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:27.823104 24193 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:27.830551 24198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:27.830791 24201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:27.831123 24199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:27.831529 24193 server_base.cc:1034] running on GCE node
I20240502 20:45:27.831900 24193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:27.832389 24193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:27.833539 24193 hybrid_clock.cc:648] HybridClock initialized: now 1714682727833523 us; error 29 us; skew 500 ppm
I20240502 20:45:27.835258 24193 webserver.cc:460] Webserver started at http://127.22.45.69:33679/ using document root <none> and password file <none>
I20240502 20:45:27.835743 24193 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:27.835795 24193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:27.835971 24193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:27.837460 24193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "d18bfcff2bbd45da8073af81a3ef6b42"
format_stamp: "Formatted at 2024-05-02 20:45:27 on dist-test-slave-g7g5"
I20240502 20:45:27.837890 24193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "d18bfcff2bbd45da8073af81a3ef6b42"
format_stamp: "Formatted at 2024-05-02 20:45:27 on dist-test-slave-g7g5"
I20240502 20:45:27.840884 24193 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 20:45:27.842888 24207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:27.843771 24193 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:45:27.843868 24193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "d18bfcff2bbd45da8073af81a3ef6b42"
format_stamp: "Formatted at 2024-05-02 20:45:27 on dist-test-slave-g7g5"
I20240502 20:45:27.843946 24193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:27.861718 24193 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:27.862291 24193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:27.862437 24193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:27.862957 24193 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:27.863776 24193 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:27.863831 24193 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:27.863883 24193 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:27.863907 24193 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:27.871563 24193 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.69:39737
I20240502 20:45:27.871641 24320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.69:39737 every 8 connection(s)
I20240502 20:45:27.872448 24193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 20:45:27.877945 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 24193
I20240502 20:45:27.878036 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240502 20:45:27.880409 24321 heartbeater.cc:347] Connected to a master server at 127.22.45.126:35441
I20240502 20:45:27.880614 24321 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:27.881011 24321 heartbeater.cc:511] Master 127.22.45.126:35441 requested a full tablet report, sending...
I20240502 20:45:27.881839 23604 ts_manager.cc:194] Registered new tserver with Master: d18bfcff2bbd45da8073af81a3ef6b42 (127.22.45.69:39737)
I20240502 20:45:27.882347 23604 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.69:51735
I20240502 20:45:27.892061 22709 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240502 20:45:27.899653 23604 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38866:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240502 20:45:27.915732 23726 tablet_service.cc:1467] Processing CreateTablet for tablet 0173f445511246e1b78a9681c037361a (DEFAULT_TABLE table=TestTable [id=fc428178bd96416ca5f8bcd1da9149d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:27.915866 23991 tablet_service.cc:1467] Processing CreateTablet for tablet 0173f445511246e1b78a9681c037361a (DEFAULT_TABLE table=TestTable [id=fc428178bd96416ca5f8bcd1da9149d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:27.916682 23991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0173f445511246e1b78a9681c037361a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:27.916692 23726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0173f445511246e1b78a9681c037361a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:27.916512 23859 tablet_service.cc:1467] Processing CreateTablet for tablet 0173f445511246e1b78a9681c037361a (DEFAULT_TABLE table=TestTable [id=fc428178bd96416ca5f8bcd1da9149d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:27.917251 23859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0173f445511246e1b78a9681c037361a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:27.921345 24338 tablet_bootstrap.cc:492] T 0173f445511246e1b78a9681c037361a P 2ecf233910ac499d91f656a4d9940dcf: Bootstrap starting.
I20240502 20:45:27.922621 24340 tablet_bootstrap.cc:492] T 0173f445511246e1b78a9681c037361a P 01ea59220a9b47e0ab797f58191bdb72: Bootstrap starting.
I20240502 20:45:27.923925 24338 tablet_bootstrap.cc:654] T 0173f445511246e1b78a9681c037361a P 2ecf233910ac499d91f656a4d9940dcf: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:27.924137 24340 tablet_bootstrap.cc:654] T 0173f445511246e1b78a9681c037361a P 01ea59220a9b47e0ab797f58191bdb72: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:27.924561 24338 log.cc:824] T 0173f445511246e1b78a9681c037361a P 2ecf233910ac499d91f656a4d9940dcf: Log is configured to *not* fsync() on all Append() calls
I20240502 20:45:27.924701 24340 log.cc:824] T 0173f445511246e1b78a9681c037361a P 01ea59220a9b47e0ab797f58191bdb72: Log is configured to *not* fsync() on all Append() calls
I20240502 20:45:27.925179 24339 tablet_bootstrap.cc:492] T 0173f445511246e1b78a9681c037361a P 57bb4cc0be2f4b6e94c119f7a39d7acf: Bootstrap starting.
I20240502 20:45:27.925997 24338 tablet_bootstrap.cc:492] T 0173f445511246e1b78a9681c037361a P 2ecf233910ac499d91f656a4d9940dcf: No bootstrap required, opened a new log
I20240502 20:45:27.926155 24338 ts_tablet_manager.cc:1367] T 0173f445511246e1b78a9681c037361a P 2ecf233910ac499d91f656a4d9940dcf: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240502 20:45:27.926164 24340 tablet_bootstrap.cc:492] T 0173f445511246e1b78a9681c037361a P 01ea59220a9b47e0ab797f58191bdb72: No bootstrap required, opened a new log
I20240502 20:45:27.926309 24340 ts_tablet_manager.cc:1367] T 0173f445511246e1b78a9681c037361a P 01ea59220a9b47e0ab797f58191bdb72: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 20:45:27.927071 24339 tablet_bootstrap.cc:654] T 0173f445511246e1b78a9681c037361a P 57bb4cc0be2f4b6e94c119f7a39d7acf: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:27.927745 24339 log.cc:824] T 0173f445511246e1b78a9681c037361a P 57bb4cc0be2f4b6e94c119f7a39d7acf: Log is configured to *not* fsync() on all Append() calls
I20240502 20:45:27.929297 24339 tablet_bootstrap.cc:492] T 0173f445511246e1b78a9681c037361a P 57bb4cc0be2f4b6e94c119f7a39d7acf: No bootstrap required, opened a new log
I20240502 20:45:27.929201 24338 raft_consensus.cc:348] T 0173f445511246e1b78a9681c037361a P 2ecf233910ac499d91f656a4d9940dcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ea59220a9b47e0ab797f58191bdb72" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46801 } } peers { permanent_uuid: "2ecf233910ac499d91f656a4d9940dcf" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 38893 } } peers { permanent_uuid: "57bb4cc0be2f4b6e94c119f7a39d7acf" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 40707 } }
I20240502 20:45:27.929365 24338 raft_consensus.cc:374] T 0173f445511246e1b78a9681c037361a P 2ecf233910ac499d91f656a4d9940dcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:27.929419 24338 raft_consensus.cc:729] T 0173f445511246e1b78a9681c037361a P 2ecf233910ac499d91f656a4d9940dcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ecf233910ac499d91f656a4d9940dcf, State: Initialized, Role: FOLLOWER
I20240502 20:45:27.929457 24339 ts_tablet_manager.cc:1367] T 0173f445511246e1b78a9681c037361a P 57bb4cc0be2f4b6e94c119f7a39d7acf: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 20:45:27.929633 24340 raft_consensus.cc:348] T 0173f445511246e1b78a9681c037361a P 01ea59220a9b47e0ab797f58191bdb72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ea59220a9b47e0ab797f58191bdb72" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46801 } } peers { permanent_uuid: "2ecf233910ac499d91f656a4d9940dcf" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 38893 } } peers { permanent_uuid: "57bb4cc0be2f4b6e94c119f7a39d7acf" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 40707 } }
I20240502 20:45:27.929841 24340 raft_consensus.cc:374] T 0173f445511246e1b78a9681c037361a P 01ea59220a9b47e0ab797f58191bdb72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:27.929792 24338 consensus_queue.cc:260] T 0173f445511246e1b78a9681c037361a P 2ecf233910ac499d91f656a4d9940dcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ea59220a9b47e0ab797f58191bdb72" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46801 } } peers { permanent_uuid: "2ecf233910ac499d91f656a4d9940dcf" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 38893 } } peers { permanent_uuid: "57bb4cc0be2f4b6e94c119f7a39d7acf" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 40707 } }
I20240502 20:45:27.929904 24340 raft_consensus.cc:729] T 0173f445511246e1b78a9681c037361a P 01ea59220a9b47e0ab797f58191bdb72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ea59220a9b47e0ab797f58191bdb72, State: Initialized, Role: FOLLOWER
I20240502 20:45:27.930292 24340 consensus_queue.cc:260] T 0173f445511246e1b78a9681c037361a P 01ea59220a9b47e0ab797f58191bdb72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ea59220a9b47e0ab797f58191bdb72" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46801 } } peers { permanent_uuid: "2ecf233910ac499d91f656a4d9940dcf" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 38893 } } peers { permanent_uuid: "57bb4cc0be2f4b6e94c119f7a39d7acf" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 40707 } }
I20240502 20:45:27.930394 24058 heartbeater.cc:503] Master 127.22.45.126:35441 was elected leader, sending a full tablet report...
I20240502 20:45:27.930622 24338 ts_tablet_manager.cc:1398] T 0173f445511246e1b78a9681c037361a P 2ecf233910ac499d91f656a4d9940dcf: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20240502 20:45:27.931128 24340 ts_tablet_manager.cc:1398] T 0173f445511246e1b78a9681c037361a P 01ea59220a9b47e0ab797f58191bdb72: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20240502 20:45:27.931460 23926 heartbeater.cc:503] Master 127.22.45.126:35441 was elected leader, sending a full tablet report...
I20240502 20:45:27.933239 24339 raft_consensus.cc:348] T 0173f445511246e1b78a9681c037361a P 57bb4cc0be2f4b6e94c119f7a39d7acf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ea59220a9b47e0ab797f58191bdb72" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46801 } } peers { permanent_uuid: "2ecf233910ac499d91f656a4d9940dcf" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 38893 } } peers { permanent_uuid: "57bb4cc0be2f4b6e94c119f7a39d7acf" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 40707 } }
I20240502 20:45:27.933394 24339 raft_consensus.cc:374] T 0173f445511246e1b78a9681c037361a P 57bb4cc0be2f4b6e94c119f7a39d7acf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:27.933434 24339 raft_consensus.cc:729] T 0173f445511246e1b78a9681c037361a P 57bb4cc0be2f4b6e94c119f7a39d7acf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57bb4cc0be2f4b6e94c119f7a39d7acf, State: Initialized, Role: FOLLOWER
I20240502 20:45:27.933753 24339 consensus_queue.cc:260] T 0173f445511246e1b78a9681c037361a P 57bb4cc0be2f4b6e94c119f7a39d7acf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ea59220a9b47e0ab797f58191bdb72" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46801 } } peers { permanent_uuid: "2ecf233910ac499d91f656a4d9940dcf" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 38893 } } peers { permanent_uuid: "57bb4cc0be2f4b6e94c119f7a39d7acf" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 40707 } }
I20240502 20:45:27.934346 24339 ts_tablet_manager.cc:1398] T 0173f445511246e1b78a9681c037361a P 57bb4cc0be2f4b6e94c119f7a39d7acf: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240502 20:45:27.934478 23794 heartbeater.cc:503] Master 127.22.45.126:35441 was elected leader, sending a full tablet report...
W20240502 20:45:27.942705 23795 tablet.cc:2369] T 0173f445511246e1b78a9681c037361a P 57bb4cc0be2f4b6e94c119f7a39d7acf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 20:45:28.011281 24344 raft_consensus.cc:482] T 0173f445511246e1b78a9681c037361a P 2ecf233910ac499d91f656a4d9940dcf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:45:28.011497 24344 raft_consensus.cc:504] T 0173f445511246e1b78a9681c037361a P 2ecf233910ac499d91f656a4d9940dcf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ea59220a9b47e0ab797f58191bdb72" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46801 } } peers { permanent_uuid: "2ecf233910ac499d91f656a4d9940dcf" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 38893 } } peers { permanent_uuid: "57bb4cc0be2f4b6e94c119f7a39d7acf" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 40707 } }
I20240502 20:45:28.013470 24344 leader_election.cc:288] T 0173f445511246e1b78a9681c037361a P 2ecf233910ac499d91f656a4d9940dcf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ea59220a9b47e0ab797f58191bdb72 (127.22.45.66:46801), 57bb4cc0be2f4b6e94c119f7a39d7acf (127.22.45.65:40707)
I20240502 20:45:28.017082 23746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0173f445511246e1b78a9681c037361a" candidate_uuid: "2ecf233910ac499d91f656a4d9940dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57bb4cc0be2f4b6e94c119f7a39d7acf" is_pre_election: true
I20240502 20:45:28.017329 23746 raft_consensus.cc:2454] T 0173f445511246e1b78a9681c037361a P 57bb4cc0be2f4b6e94c119f7a39d7acf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ecf233910ac499d91f656a4d9940dcf in term 0.
I20240502 20:45:28.017802 23944 leader_election.cc:302] T 0173f445511246e1b78a9681c037361a P 2ecf233910ac499d91f656a4d9940dcf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ecf233910ac499d91f656a4d9940dcf, 57bb4cc0be2f4b6e94c119f7a39d7acf; no voters:
I20240502 20:45:28.018043 24344 raft_consensus.cc:2781] T 0173f445511246e1b78a9681c037361a P 2ecf233910ac499d91f656a4d9940dcf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:45:28.018013 23879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0173f445511246e1b78a9681c037361a" candidate_uuid: "2ecf233910ac499d91f656a4d9940dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ea59220a9b47e0ab797f58191bdb72" is_pre_election: true
I20240502 20:45:28.018110 24344 raft_consensus.cc:482] T 0173f445511246e1b78a9681c037361a P 2ecf233910ac499d91f656a4d9940dcf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:45:28.018154 24344 raft_consensus.cc:3037] T 0173f445511246e1b78a9681c037361a P 2ecf233910ac499d91f656a4d9940dcf [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:28.018275 23879 raft_consensus.cc:2454] T 0173f445511246e1b78a9681c037361a P 01ea59220a9b47e0ab797f58191bdb72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ecf233910ac499d91f656a4d9940dcf in term 0.
W20240502 20:45:28.018801 24059 tablet.cc:2369] T 0173f445511246e1b78a9681c037361a P 2ecf233910ac499d91f656a4d9940dcf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 20:45:28.019100 24344 raft_consensus.cc:504] T 0173f445511246e1b78a9681c037361a P 2ecf233910ac499d91f656a4d9940dcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ea59220a9b47e0ab797f58191bdb72" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46801 } } peers { permanent_uuid: "2ecf233910ac499d91f656a4d9940dcf" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 38893 } } peers { permanent_uuid: "57bb4cc0be2f4b6e94c119f7a39d7acf" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 40707 } }
I20240502 20:45:28.019399 24344 leader_election.cc:288] T 0173f445511246e1b78a9681c037361a P 2ecf233910ac499d91f656a4d9940dcf [CANDIDATE]: Term 1 election: Requested vote from peers 01ea59220a9b47e0ab797f58191bdb72 (127.22.45.66:46801), 57bb4cc0be2f4b6e94c119f7a39d7acf (127.22.45.65:40707)
I20240502 20:45:28.019706 23879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0173f445511246e1b78a9681c037361a" candidate_uuid: "2ecf233910ac499d91f656a4d9940dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ea59220a9b47e0ab797f58191bdb72"
I20240502 20:45:28.019742 23746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0173f445511246e1b78a9681c037361a" candidate_uuid: "2ecf233910ac499d91f656a4d9940dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57bb4cc0be2f4b6e94c119f7a39d7acf"
I20240502 20:45:28.019800 23879 raft_consensus.cc:3037] T 0173f445511246e1b78a9681c037361a P 01ea59220a9b47e0ab797f58191bdb72 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:28.019867 23746 raft_consensus.cc:3037] T 0173f445511246e1b78a9681c037361a P 57bb4cc0be2f4b6e94c119f7a39d7acf [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:28.020615 23879 raft_consensus.cc:2454] T 0173f445511246e1b78a9681c037361a P 01ea59220a9b47e0ab797f58191bdb72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ecf233910ac499d91f656a4d9940dcf in term 1.
I20240502 20:45:28.020701 23746 raft_consensus.cc:2454] T 0173f445511246e1b78a9681c037361a P 57bb4cc0be2f4b6e94c119f7a39d7acf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ecf233910ac499d91f656a4d9940dcf in term 1.
I20240502 20:45:28.020896 23946 leader_election.cc:302] T 0173f445511246e1b78a9681c037361a P 2ecf233910ac499d91f656a4d9940dcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ea59220a9b47e0ab797f58191bdb72, 2ecf233910ac499d91f656a4d9940dcf; no voters:
I20240502 20:45:28.021081 24344 raft_consensus.cc:2781] T 0173f445511246e1b78a9681c037361a P 2ecf233910ac499d91f656a4d9940dcf [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:45:28.021239 24344 raft_consensus.cc:686] T 0173f445511246e1b78a9681c037361a P 2ecf233910ac499d91f656a4d9940dcf [term 1 LEADER]: Becoming Leader. State: Replica: 2ecf233910ac499d91f656a4d9940dcf, State: Running, Role: LEADER
I20240502 20:45:28.021498 24344 consensus_queue.cc:237] T 0173f445511246e1b78a9681c037361a P 2ecf233910ac499d91f656a4d9940dcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ea59220a9b47e0ab797f58191bdb72" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46801 } } peers { permanent_uuid: "2ecf233910ac499d91f656a4d9940dcf" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 38893 } } peers { permanent_uuid: "57bb4cc0be2f4b6e94c119f7a39d7acf" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 40707 } }
I20240502 20:45:28.024148 23602 catalog_manager.cc:5484] T 0173f445511246e1b78a9681c037361a P 2ecf233910ac499d91f656a4d9940dcf reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ecf233910ac499d91f656a4d9940dcf (127.22.45.67). New cstate: current_term: 1 leader_uuid: "2ecf233910ac499d91f656a4d9940dcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ea59220a9b47e0ab797f58191bdb72" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46801 } } peers { permanent_uuid: "2ecf233910ac499d91f656a4d9940dcf" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 38893 } } peers { permanent_uuid: "57bb4cc0be2f4b6e94c119f7a39d7acf" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 40707 } } }
I20240502 20:45:28.031455 22709 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240502 20:45:28.032763 22709 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 57bb4cc0be2f4b6e94c119f7a39d7acf to finish bootstrapping
I20240502 20:45:28.037374 22709 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 01ea59220a9b47e0ab797f58191bdb72 to finish bootstrapping
I20240502 20:45:28.041448 22709 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 2ecf233910ac499d91f656a4d9940dcf to finish bootstrapping
I20240502 20:45:28.045392 22709 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 70053cc6a9b74bf18b71345465585c60 to finish bootstrapping
I20240502 20:45:28.050494 22709 ts_itest-base.cc:233] Waiting for 0 tablets on tserver d18bfcff2bbd45da8073af81a3ef6b42 to finish bootstrapping
I20240502 20:45:28.055323 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 23664
W20240502 20:45:28.105290 23927 tablet.cc:2369] T 0173f445511246e1b78a9681c037361a P 01ea59220a9b47e0ab797f58191bdb72: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 20:45:28.231151 23859 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:28.231832 23991 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:28.231983 24123 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:28.232789 24255 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:28.299539 23991 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:28.299698 24123 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:28.300138 23859 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:28.308012 24255 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:28.398577 23859 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:28.399277 24123 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:28.399950 23991 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:28.400208 24255 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:28.506587 24344 consensus_queue.cc:1035] T 0173f445511246e1b78a9681c037361a P 2ecf233910ac499d91f656a4d9940dcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ea59220a9b47e0ab797f58191bdb72" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 20:45:28.510768 23944 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.45.65:40707: connect: Connection refused (error 111)
W20240502 20:45:28.511818 23944 consensus_peers.cc:485] T 0173f445511246e1b78a9681c037361a P 2ecf233910ac499d91f656a4d9940dcf -> Peer 57bb4cc0be2f4b6e94c119f7a39d7acf (127.22.45.65:40707): Couldn't send request to peer 57bb4cc0be2f4b6e94c119f7a39d7acf. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.65:40707: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 20:45:28.607875 23991 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:28.608157 23859 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:28.608683 24255 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:28.609763 24123 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:28.674723 23859 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:28.674959 24123 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:28.675460 23991 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:28.675510 24255 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:28.718994 24190 heartbeater.cc:503] Master 127.22.45.126:35441 was elected leader, sending a full tablet report...
I20240502 20:45:28.773823 24123 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:28.773900 23859 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:28.774204 23991 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:28.779196 24255 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:28.810274 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 23797
I20240502 20:45:28.815205 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 23929
I20240502 20:45:28.819859 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 24061
I20240502 20:45:28.824235 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 24193
I20240502 20:45:28.828519 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 23573
2024-05-02T20:45:28Z chronyd exiting
[ OK ] RebalanceStartCriteriaTest.TabletServerIsDown/0 (2058 ms)
[----------] 1 test from RebalanceStartCriteriaTest (2058 ms total)
[----------] 1 test from RebalanceParamTest
[ RUN ] RebalanceParamTest.Rebalance/0
I20240502 20:45:28.849118 22709 test_util.cc:268] Using random seed: -1359700261
I20240502 20:45:28.849812 22709 ts_itest-base.cc:112] Starting cluster with:
I20240502 20:45:28.849850 22709 ts_itest-base.cc:113] --------------
I20240502 20:45:28.849860 22709 ts_itest-base.cc:114] 7 tablet servers
I20240502 20:45:28.849869 22709 ts_itest-base.cc:115] 1 replicas per TS
I20240502 20:45:28.849880 22709 ts_itest-base.cc:116] --------------
2024-05-02T20:45:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T20:45:28Z Disabled control of system clock
I20240502 20:45:28.857004 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.45.126:34337
--webserver_interface=127.22.45.126
--webserver_port=0
--builtin_ntp_servers=127.22.45.84:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.45.126:34337
--allow_unsafe_replication_factor
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=false
--tserver_unresponsive_timeout_ms=3000 with env {}
W20240502 20:45:28.956625 24542 flags.cc:425] Enabled unsafe flag: --allow_unsafe_replication_factor=true
W20240502 20:45:28.956835 24542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:28.956859 24542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:28.956900 24542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:28.959784 24542 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:45:28.959841 24542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 20:45:28.959868 24542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:28.959888 24542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 20:45:28.959910 24542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 20:45:28.963446 24542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal
--allow_unsafe_replication_factor=true
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.22.45.126:34337
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.45.126:34337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.22.45.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.24542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:28.964188 24542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:28.965176 24542 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:28.971997 24548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:28.972229 24550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:28.972617 24547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:28.973104 24542 server_base.cc:1034] running on GCE node
I20240502 20:45:28.973517 24542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:28.974313 24542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:28.975456 24542 hybrid_clock.cc:648] HybridClock initialized: now 1714682728975447 us; error 38 us; skew 500 ppm
I20240502 20:45:28.977174 24542 webserver.cc:460] Webserver started at http://127.22.45.126:41127/ using document root <none> and password file <none>
I20240502 20:45:28.977665 24542 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:28.977715 24542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:28.977861 24542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:28.979403 24542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "fe9820ffe81d4edb865d80b0d7faae0c"
format_stamp: "Formatted at 2024-05-02 20:45:28 on dist-test-slave-g7g5"
I20240502 20:45:28.979823 24542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "fe9820ffe81d4edb865d80b0d7faae0c"
format_stamp: "Formatted at 2024-05-02 20:45:28 on dist-test-slave-g7g5"
I20240502 20:45:28.982790 24542 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 20:45:28.984714 24556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:28.985626 24542 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:45:28.985726 24542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal
uuid: "fe9820ffe81d4edb865d80b0d7faae0c"
format_stamp: "Formatted at 2024-05-02 20:45:28 on dist-test-slave-g7g5"
I20240502 20:45:28.985796 24542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:28.996445 24542 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:28.996970 24542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:28.997090 24542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:29.003010 24542 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.126:34337
I20240502 20:45:29.003007 24609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.126:34337 every 8 connection(s)
I20240502 20:45:29.003888 24542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 20:45:29.006192 24610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:29.010779 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 24542
I20240502 20:45:29.010898 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 20:45:29.011050 24610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe9820ffe81d4edb865d80b0d7faae0c: Bootstrap starting.
I20240502 20:45:29.013268 24610 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe9820ffe81d4edb865d80b0d7faae0c: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:29.013906 24610 log.cc:824] T 00000000000000000000000000000000 P fe9820ffe81d4edb865d80b0d7faae0c: Log is configured to *not* fsync() on all Append() calls
I20240502 20:45:29.015359 24610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe9820ffe81d4edb865d80b0d7faae0c: No bootstrap required, opened a new log
I20240502 20:45:29.017848 24610 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe9820ffe81d4edb865d80b0d7faae0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe9820ffe81d4edb865d80b0d7faae0c" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 34337 } }
I20240502 20:45:29.018059 24610 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe9820ffe81d4edb865d80b0d7faae0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:29.018116 24610 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe9820ffe81d4edb865d80b0d7faae0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe9820ffe81d4edb865d80b0d7faae0c, State: Initialized, Role: FOLLOWER
I20240502 20:45:29.018550 24610 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe9820ffe81d4edb865d80b0d7faae0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe9820ffe81d4edb865d80b0d7faae0c" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 34337 } }
I20240502 20:45:29.018669 24610 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe9820ffe81d4edb865d80b0d7faae0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 20:45:29.018726 24610 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe9820ffe81d4edb865d80b0d7faae0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 20:45:29.018831 24610 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe9820ffe81d4edb865d80b0d7faae0c [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:29.019582 24610 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe9820ffe81d4edb865d80b0d7faae0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe9820ffe81d4edb865d80b0d7faae0c" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 34337 } }
I20240502 20:45:29.019876 24610 leader_election.cc:302] T 00000000000000000000000000000000 P fe9820ffe81d4edb865d80b0d7faae0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe9820ffe81d4edb865d80b0d7faae0c; no voters:
I20240502 20:45:29.020112 24610 leader_election.cc:288] T 00000000000000000000000000000000 P fe9820ffe81d4edb865d80b0d7faae0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 20:45:29.020423 24615 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe9820ffe81d4edb865d80b0d7faae0c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:45:29.020599 24615 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe9820ffe81d4edb865d80b0d7faae0c [term 1 LEADER]: Becoming Leader. State: Replica: fe9820ffe81d4edb865d80b0d7faae0c, State: Running, Role: LEADER
I20240502 20:45:29.020843 24615 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe9820ffe81d4edb865d80b0d7faae0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe9820ffe81d4edb865d80b0d7faae0c" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 34337 } }
I20240502 20:45:29.021090 24610 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe9820ffe81d4edb865d80b0d7faae0c [sys.catalog]: configured and running, proceeding with master startup.
I20240502 20:45:29.022193 24617 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe9820ffe81d4edb865d80b0d7faae0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe9820ffe81d4edb865d80b0d7faae0c. Latest consensus state: current_term: 1 leader_uuid: "fe9820ffe81d4edb865d80b0d7faae0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe9820ffe81d4edb865d80b0d7faae0c" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 34337 } } }
I20240502 20:45:29.022193 24616 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe9820ffe81d4edb865d80b0d7faae0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe9820ffe81d4edb865d80b0d7faae0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe9820ffe81d4edb865d80b0d7faae0c" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 34337 } } }
I20240502 20:45:29.022420 24616 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe9820ffe81d4edb865d80b0d7faae0c [sys.catalog]: This master's current role is: LEADER
I20240502 20:45:29.022449 24617 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe9820ffe81d4edb865d80b0d7faae0c [sys.catalog]: This master's current role is: LEADER
W20240502 20:45:29.025223 24630 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fe9820ffe81d4edb865d80b0d7faae0c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240502 20:45:29.025283 24630 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20240502 20:45:29.025364 24626 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 20:45:29.026275 24626 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 20:45:29.030756 24626 catalog_manager.cc:1347] Generated new cluster ID: 285f6defa3e542cbb2482a7b7114db1d
I20240502 20:45:29.030817 24626 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 20:45:29.064289 24626 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 20:45:29.065194 24626 catalog_manager.cc:1504] Loading token signing keys...
I20240502 20:45:29.075383 24626 catalog_manager.cc:5857] T 00000000000000000000000000000000 P fe9820ffe81d4edb865d80b0d7faae0c: Generated new TSK 0
I20240502 20:45:29.076026 24626 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 20:45:29.081725 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.65:0
--local_ip_for_outbound_sockets=127.22.45.65
--webserver_interface=127.22.45.65
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34337
--builtin_ntp_servers=127.22.45.84:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 20:45:29.184928 24634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:29.185117 24634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:29.185163 24634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:29.188158 24634 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:45:29.188216 24634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:29.188297 24634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.65
I20240502 20:45:29.191838 24634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.22.45.65
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34337
--never_fsync=true
--heap_profile_path=/tmp/kudu.24634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.65
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:29.192664 24634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:29.193732 24634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:29.201279 24640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:29.201530 24642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:29.201906 24639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:29.202266 24634 server_base.cc:1034] running on GCE node
I20240502 20:45:29.202649 24634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:29.203145 24634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:29.204301 24634 hybrid_clock.cc:648] HybridClock initialized: now 1714682729204287 us; error 26 us; skew 500 ppm
I20240502 20:45:29.205893 24634 webserver.cc:460] Webserver started at http://127.22.45.65:38633/ using document root <none> and password file <none>
I20240502 20:45:29.206403 24634 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:29.206478 24634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:29.206643 24634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:29.208180 24634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "1c52358e8e2f4ffe992cafe0f1a95a77"
format_stamp: "Formatted at 2024-05-02 20:45:29 on dist-test-slave-g7g5"
I20240502 20:45:29.208606 24634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "1c52358e8e2f4ffe992cafe0f1a95a77"
format_stamp: "Formatted at 2024-05-02 20:45:29 on dist-test-slave-g7g5"
I20240502 20:45:29.211630 24634 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20240502 20:45:29.213618 24648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:29.214562 24634 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240502 20:45:29.214665 24634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "1c52358e8e2f4ffe992cafe0f1a95a77"
format_stamp: "Formatted at 2024-05-02 20:45:29 on dist-test-slave-g7g5"
I20240502 20:45:29.214740 24634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:29.231864 24634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:29.232451 24634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:29.232599 24634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:29.233111 24634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:29.233932 24634 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:29.233978 24634 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:29.234027 24634 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:29.234061 24634 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:29.242014 24634 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.65:36905
I20240502 20:45:29.242050 24762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.65:36905 every 8 connection(s)
I20240502 20:45:29.242882 24634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 20:45:29.247040 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 24634
I20240502 20:45:29.247138 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 20:45:29.249256 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.66:0
--local_ip_for_outbound_sockets=127.22.45.66
--webserver_interface=127.22.45.66
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34337
--builtin_ntp_servers=127.22.45.84:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:45:29.252354 24763 heartbeater.cc:347] Connected to a master server at 127.22.45.126:34337
I20240502 20:45:29.252559 24763 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:29.252990 24763 heartbeater.cc:511] Master 127.22.45.126:34337 requested a full tablet report, sending...
I20240502 20:45:29.254274 24573 ts_manager.cc:194] Registered new tserver with Master: 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65:36905)
I20240502 20:45:29.255407 24573 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.65:52709
W20240502 20:45:29.350140 24766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:29.350347 24766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:29.350405 24766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:29.353344 24766 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:45:29.353410 24766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:29.353492 24766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.66
I20240502 20:45:29.357106 24766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.22.45.66
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34337
--never_fsync=true
--heap_profile_path=/tmp/kudu.24766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.66
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:29.357939 24766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:29.359038 24766 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:29.366581 24771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:29.366784 24772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:29.367076 24774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:29.367549 24766 server_base.cc:1034] running on GCE node
I20240502 20:45:29.367869 24766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:29.368342 24766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:29.369495 24766 hybrid_clock.cc:648] HybridClock initialized: now 1714682729369480 us; error 31 us; skew 500 ppm
I20240502 20:45:29.371238 24766 webserver.cc:460] Webserver started at http://127.22.45.66:34159/ using document root <none> and password file <none>
I20240502 20:45:29.371734 24766 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:29.371790 24766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:29.371958 24766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:29.373495 24766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "f926d9d9d2314c2b90437934356f66d4"
format_stamp: "Formatted at 2024-05-02 20:45:29 on dist-test-slave-g7g5"
I20240502 20:45:29.373915 24766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "f926d9d9d2314c2b90437934356f66d4"
format_stamp: "Formatted at 2024-05-02 20:45:29 on dist-test-slave-g7g5"
I20240502 20:45:29.376960 24766 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20240502 20:45:29.378933 24780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:29.379843 24766 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:45:29.379951 24766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "f926d9d9d2314c2b90437934356f66d4"
format_stamp: "Formatted at 2024-05-02 20:45:29 on dist-test-slave-g7g5"
I20240502 20:45:29.380026 24766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:29.406545 24766 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:29.407141 24766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:29.407287 24766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:29.407784 24766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:29.408634 24766 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:29.408680 24766 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:29.408730 24766 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:29.408758 24766 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:29.416350 24766 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.66:46041
I20240502 20:45:29.416399 24893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.66:46041 every 8 connection(s)
I20240502 20:45:29.417382 24766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 20:45:29.424726 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 24766
I20240502 20:45:29.424850 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 20:45:29.426442 24894 heartbeater.cc:347] Connected to a master server at 127.22.45.126:34337
I20240502 20:45:29.426687 24894 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:29.427011 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.67:0
--local_ip_for_outbound_sockets=127.22.45.67
--webserver_interface=127.22.45.67
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34337
--builtin_ntp_servers=127.22.45.84:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:45:29.427196 24894 heartbeater.cc:511] Master 127.22.45.126:34337 requested a full tablet report, sending...
I20240502 20:45:29.428139 24573 ts_manager.cc:194] Registered new tserver with Master: f926d9d9d2314c2b90437934356f66d4 (127.22.45.66:46041)
I20240502 20:45:29.428736 24573 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.66:57763
W20240502 20:45:29.527562 24897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:29.527752 24897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:29.527810 24897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:29.530746 24897 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:45:29.530807 24897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:29.530885 24897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.67
I20240502 20:45:29.534368 24897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.22.45.67
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34337
--never_fsync=true
--heap_profile_path=/tmp/kudu.24897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.67
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:29.535189 24897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:29.536229 24897 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:29.543740 24902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:29.544189 24905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:29.544523 24903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:29.544941 24897 server_base.cc:1034] running on GCE node
I20240502 20:45:29.545349 24897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:29.545881 24897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:29.547024 24897 hybrid_clock.cc:648] HybridClock initialized: now 1714682729547014 us; error 26 us; skew 500 ppm
I20240502 20:45:29.548601 24897 webserver.cc:460] Webserver started at http://127.22.45.67:44427/ using document root <none> and password file <none>
I20240502 20:45:29.549108 24897 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:29.549180 24897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:29.549343 24897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:29.550985 24897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "a984b4b59cb6429290bc907a860d50f8"
format_stamp: "Formatted at 2024-05-02 20:45:29 on dist-test-slave-g7g5"
I20240502 20:45:29.551388 24897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "a984b4b59cb6429290bc907a860d50f8"
format_stamp: "Formatted at 2024-05-02 20:45:29 on dist-test-slave-g7g5"
I20240502 20:45:29.554345 24897 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 20:45:29.556317 24911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:29.557238 24897 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:45:29.557333 24897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "a984b4b59cb6429290bc907a860d50f8"
format_stamp: "Formatted at 2024-05-02 20:45:29 on dist-test-slave-g7g5"
I20240502 20:45:29.557407 24897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:29.576833 24897 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:29.577387 24897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:29.577526 24897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:29.578008 24897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:29.578861 24897 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:29.578905 24897 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:29.578961 24897 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:29.578984 24897 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:29.586812 24897 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.67:44809
I20240502 20:45:29.586858 25024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.67:44809 every 8 connection(s)
I20240502 20:45:29.587642 24897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 20:45:29.592422 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 24897
I20240502 20:45:29.592540 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 20:45:29.594718 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.68:0
--local_ip_for_outbound_sockets=127.22.45.68
--webserver_interface=127.22.45.68
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34337
--builtin_ntp_servers=127.22.45.84:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:45:29.596050 25025 heartbeater.cc:347] Connected to a master server at 127.22.45.126:34337
I20240502 20:45:29.596275 25025 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:29.596751 25025 heartbeater.cc:511] Master 127.22.45.126:34337 requested a full tablet report, sending...
I20240502 20:45:29.597645 24573 ts_manager.cc:194] Registered new tserver with Master: a984b4b59cb6429290bc907a860d50f8 (127.22.45.67:44809)
I20240502 20:45:29.598143 24573 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.67:51533
W20240502 20:45:29.695109 25028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:29.695297 25028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:29.695353 25028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:29.698251 25028 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:45:29.698313 25028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:29.698395 25028 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.68
I20240502 20:45:29.701836 25028 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.22.45.68
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34337
--never_fsync=true
--heap_profile_path=/tmp/kudu.25028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.68
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:29.702634 25028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:29.703653 25028 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:29.710965 25034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:29.711220 25033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:29.711596 25036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:29.712054 25028 server_base.cc:1034] running on GCE node
I20240502 20:45:29.712360 25028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:29.712865 25028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:29.714015 25028 hybrid_clock.cc:648] HybridClock initialized: now 1714682729713993 us; error 30 us; skew 500 ppm
I20240502 20:45:29.715651 25028 webserver.cc:460] Webserver started at http://127.22.45.68:41093/ using document root <none> and password file <none>
I20240502 20:45:29.716145 25028 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:29.716198 25028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:29.716364 25028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:29.717881 25028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "bb697f05a8394524a411ece19fe608dc"
format_stamp: "Formatted at 2024-05-02 20:45:29 on dist-test-slave-g7g5"
I20240502 20:45:29.718336 25028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "bb697f05a8394524a411ece19fe608dc"
format_stamp: "Formatted at 2024-05-02 20:45:29 on dist-test-slave-g7g5"
I20240502 20:45:29.721390 25028 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240502 20:45:29.723400 25042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:29.724304 25028 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 20:45:29.724408 25028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "bb697f05a8394524a411ece19fe608dc"
format_stamp: "Formatted at 2024-05-02 20:45:29 on dist-test-slave-g7g5"
I20240502 20:45:29.724483 25028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:29.738898 25028 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:29.739425 25028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:29.739559 25028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:29.740020 25028 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:29.740799 25028 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:29.740844 25028 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:29.740891 25028 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:29.740938 25028 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:29.748777 25028 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.68:39501
I20240502 20:45:29.748809 25155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.68:39501 every 8 connection(s)
I20240502 20:45:29.749799 25028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 20:45:29.758139 25156 heartbeater.cc:347] Connected to a master server at 127.22.45.126:34337
I20240502 20:45:29.758311 25156 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:29.758708 25156 heartbeater.cc:511] Master 127.22.45.126:34337 requested a full tablet report, sending...
I20240502 20:45:29.759546 24573 ts_manager.cc:194] Registered new tserver with Master: bb697f05a8394524a411ece19fe608dc (127.22.45.68:39501)
I20240502 20:45:29.759851 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 25028
I20240502 20:45:29.759968 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 20:45:29.760126 24573 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.68:55647
I20240502 20:45:29.761850 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.69:0
--local_ip_for_outbound_sockets=127.22.45.69
--webserver_interface=127.22.45.69
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34337
--builtin_ntp_servers=127.22.45.84:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 20:45:29.870968 25159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:29.871146 25159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:29.871186 25159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:29.874338 25159 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:45:29.874397 25159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:29.874485 25159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.69
I20240502 20:45:29.878015 25159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.69:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.22.45.69
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34337
--never_fsync=true
--heap_profile_path=/tmp/kudu.25159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.69
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:29.878809 25159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:29.879818 25159 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:29.887225 25165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:29.887511 25164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:29.887907 25167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:29.888367 25159 server_base.cc:1034] running on GCE node
I20240502 20:45:29.888689 25159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:29.889175 25159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:29.890326 25159 hybrid_clock.cc:648] HybridClock initialized: now 1714682729890312 us; error 27 us; skew 500 ppm
I20240502 20:45:29.891897 25159 webserver.cc:460] Webserver started at http://127.22.45.69:37115/ using document root <none> and password file <none>
I20240502 20:45:29.892390 25159 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:29.892446 25159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:29.892606 25159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:29.894129 25159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "eef5daac82024c449fc8b0dc82bd428c"
format_stamp: "Formatted at 2024-05-02 20:45:29 on dist-test-slave-g7g5"
I20240502 20:45:29.894595 25159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "eef5daac82024c449fc8b0dc82bd428c"
format_stamp: "Formatted at 2024-05-02 20:45:29 on dist-test-slave-g7g5"
I20240502 20:45:29.897601 25159 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240502 20:45:29.899561 25173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:29.900452 25159 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 20:45:29.900557 25159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "eef5daac82024c449fc8b0dc82bd428c"
format_stamp: "Formatted at 2024-05-02 20:45:29 on dist-test-slave-g7g5"
I20240502 20:45:29.900630 25159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:29.924791 25159 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:29.925375 25159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:29.925516 25159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:29.926002 25159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:29.926913 25159 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:29.926960 25159 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:29.927029 25159 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:29.927059 25159 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:29.934839 25159 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.69:37677
I20240502 20:45:29.934864 25286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.69:37677 every 8 connection(s)
I20240502 20:45:29.935667 25159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 20:45:29.936614 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 25159
I20240502 20:45:29.936722 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240502 20:45:29.938758 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.70:0
--local_ip_for_outbound_sockets=127.22.45.70
--webserver_interface=127.22.45.70
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34337
--builtin_ntp_servers=127.22.45.84:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:45:29.944172 25287 heartbeater.cc:347] Connected to a master server at 127.22.45.126:34337
I20240502 20:45:29.944375 25287 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:29.945255 25287 heartbeater.cc:511] Master 127.22.45.126:34337 requested a full tablet report, sending...
I20240502 20:45:29.946197 24573 ts_manager.cc:194] Registered new tserver with Master: eef5daac82024c449fc8b0dc82bd428c (127.22.45.69:37677)
I20240502 20:45:29.946926 24573 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.69:48281
W20240502 20:45:30.043288 25290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:30.043466 25290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:30.043505 25290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:30.046366 25290 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:45:30.046418 25290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:30.046512 25290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.70
I20240502 20:45:30.049990 25290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.70:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.22.45.70
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34337
--never_fsync=true
--heap_profile_path=/tmp/kudu.25290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.70
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:30.050753 25290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:30.051735 25290 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:30.059195 25296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:30.059523 25295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:30.059917 25298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:30.060303 25290 server_base.cc:1034] running on GCE node
I20240502 20:45:30.060601 25290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:30.061093 25290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:30.062263 25290 hybrid_clock.cc:648] HybridClock initialized: now 1714682730062247 us; error 34 us; skew 500 ppm
I20240502 20:45:30.063887 25290 webserver.cc:460] Webserver started at http://127.22.45.70:33401/ using document root <none> and password file <none>
I20240502 20:45:30.064385 25290 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:30.064440 25290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:30.064595 25290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:30.066094 25290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "f6dd93d055454bc5965b407745c020ff"
format_stamp: "Formatted at 2024-05-02 20:45:30 on dist-test-slave-g7g5"
I20240502 20:45:30.066555 25290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "f6dd93d055454bc5965b407745c020ff"
format_stamp: "Formatted at 2024-05-02 20:45:30 on dist-test-slave-g7g5"
I20240502 20:45:30.069509 25290 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20240502 20:45:30.071468 25304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:30.072352 25290 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 20:45:30.072449 25290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "f6dd93d055454bc5965b407745c020ff"
format_stamp: "Formatted at 2024-05-02 20:45:30 on dist-test-slave-g7g5"
I20240502 20:45:30.072520 25290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:30.086822 25290 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:30.087343 25290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:30.087473 25290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:30.087922 25290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:30.088762 25290 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:30.088809 25290 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:30.088855 25290 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:30.088881 25290 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:30.097275 25290 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.70:36617
I20240502 20:45:30.097292 25418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.70:36617 every 8 connection(s)
I20240502 20:45:30.098068 25290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 20:45:30.104846 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 25290
I20240502 20:45:30.104983 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240502 20:45:30.105954 25419 heartbeater.cc:347] Connected to a master server at 127.22.45.126:34337
I20240502 20:45:30.106153 25419 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:30.106560 25419 heartbeater.cc:511] Master 127.22.45.126:34337 requested a full tablet report, sending...
I20240502 20:45:30.106957 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.71:0
--local_ip_for_outbound_sockets=127.22.45.71
--webserver_interface=127.22.45.71
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34337
--builtin_ntp_servers=127.22.45.84:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:45:30.107446 24573 ts_manager.cc:194] Registered new tserver with Master: f6dd93d055454bc5965b407745c020ff (127.22.45.70:36617)
I20240502 20:45:30.107964 24573 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.70:46103
W20240502 20:45:30.208371 25422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:30.208554 25422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:30.208591 25422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:30.211488 25422 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:45:30.211539 25422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:30.211613 25422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.71
I20240502 20:45:30.215029 25422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.71:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.22.45.71
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34337
--never_fsync=true
--heap_profile_path=/tmp/kudu.25422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.71
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:30.215797 25422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:30.216814 25422 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:30.224575 25428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:30.224840 25427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:30.225219 25430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:30.225616 25422 server_base.cc:1034] running on GCE node
I20240502 20:45:30.225936 25422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:30.226421 25422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:30.227581 25422 hybrid_clock.cc:648] HybridClock initialized: now 1714682730227564 us; error 30 us; skew 500 ppm
I20240502 20:45:30.229178 25422 webserver.cc:460] Webserver started at http://127.22.45.71:39827/ using document root <none> and password file <none>
I20240502 20:45:30.229665 25422 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:30.229722 25422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:30.229878 25422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:30.231436 25422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "85b4f683f6904c35b3c0e91799295bb3"
format_stamp: "Formatted at 2024-05-02 20:45:30 on dist-test-slave-g7g5"
I20240502 20:45:30.231873 25422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "85b4f683f6904c35b3c0e91799295bb3"
format_stamp: "Formatted at 2024-05-02 20:45:30 on dist-test-slave-g7g5"
I20240502 20:45:30.234858 25422 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20240502 20:45:30.236800 25436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:30.237694 25422 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:45:30.237790 25422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "85b4f683f6904c35b3c0e91799295bb3"
format_stamp: "Formatted at 2024-05-02 20:45:30 on dist-test-slave-g7g5"
I20240502 20:45:30.237864 25422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:30.257385 24763 heartbeater.cc:503] Master 127.22.45.126:34337 was elected leader, sending a full tablet report...
I20240502 20:45:30.262101 25422 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:30.262754 25422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:30.262897 25422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:30.263407 25422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:30.264252 25422 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:30.264307 25422 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:30.264357 25422 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:30.264380 25422 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:30.272320 25422 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.71:39125
I20240502 20:45:30.272337 25550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.71:39125 every 8 connection(s)
I20240502 20:45:30.273201 25422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 20:45:30.281265 25551 heartbeater.cc:347] Connected to a master server at 127.22.45.126:34337
I20240502 20:45:30.281469 25551 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:30.281903 25551 heartbeater.cc:511] Master 127.22.45.126:34337 requested a full tablet report, sending...
I20240502 20:45:30.282584 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 25422
I20240502 20:45:30.282688 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240502 20:45:30.282900 24573 ts_manager.cc:194] Registered new tserver with Master: 85b4f683f6904c35b3c0e91799295bb3 (127.22.45.71:39125)
I20240502 20:45:30.283546 24573 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.71:37115
I20240502 20:45:30.284801 22709 external_mini_cluster.cc:930] 7 TS(s) registered with all masters
I20240502 20:45:30.293133 24573 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46840:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240502 20:45:30.308346 24959 tablet_service.cc:1467] Processing CreateTablet for tablet dec4ab92da444ee28d3c9ccaa498cb5e (DEFAULT_TABLE table=TestTable [id=c5ab41672bde4728aab7b987bd50b139]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:30.309192 24959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dec4ab92da444ee28d3c9ccaa498cb5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:30.313939 25565 tablet_bootstrap.cc:492] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8: Bootstrap starting.
I20240502 20:45:30.316318 25565 tablet_bootstrap.cc:654] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:30.317090 25565 log.cc:824] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8: Log is configured to *not* fsync() on all Append() calls
I20240502 20:45:30.319021 25565 tablet_bootstrap.cc:492] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8: No bootstrap required, opened a new log
I20240502 20:45:30.319197 25565 ts_tablet_manager.cc:1367] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240502 20:45:30.321595 25565 raft_consensus.cc:348] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a984b4b59cb6429290bc907a860d50f8" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44809 } }
I20240502 20:45:30.321761 25565 raft_consensus.cc:374] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:30.321811 25565 raft_consensus.cc:729] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a984b4b59cb6429290bc907a860d50f8, State: Initialized, Role: FOLLOWER
I20240502 20:45:30.322186 25565 consensus_queue.cc:260] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a984b4b59cb6429290bc907a860d50f8" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44809 } }
I20240502 20:45:30.322333 25565 raft_consensus.cc:388] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 20:45:30.322394 25565 raft_consensus.cc:482] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 20:45:30.322518 25565 raft_consensus.cc:3037] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:30.415179 25565 raft_consensus.cc:504] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a984b4b59cb6429290bc907a860d50f8" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44809 } }
I20240502 20:45:30.415699 25565 leader_election.cc:302] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a984b4b59cb6429290bc907a860d50f8; no voters:
I20240502 20:45:30.416116 25565 leader_election.cc:288] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 20:45:30.416307 25567 raft_consensus.cc:2781] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:45:30.416513 25567 raft_consensus.cc:686] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8 [term 1 LEADER]: Becoming Leader. State: Replica: a984b4b59cb6429290bc907a860d50f8, State: Running, Role: LEADER
I20240502 20:45:30.416821 25567 consensus_queue.cc:237] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a984b4b59cb6429290bc907a860d50f8" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44809 } }
I20240502 20:45:30.417047 25565 ts_tablet_manager.cc:1398] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8: Time spent starting tablet: real 0.098s user 0.006s sys 0.000s
I20240502 20:45:30.417451 25025 heartbeater.cc:503] Master 127.22.45.126:34337 was elected leader, sending a full tablet report...
I20240502 20:45:30.420413 24573 catalog_manager.cc:5484] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a984b4b59cb6429290bc907a860d50f8 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "a984b4b59cb6429290bc907a860d50f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a984b4b59cb6429290bc907a860d50f8" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44809 } } }
I20240502 20:45:30.430404 24894 heartbeater.cc:503] Master 127.22.45.126:34337 was elected leader, sending a full tablet report...
I20240502 20:45:30.452769 22709 external_mini_cluster.cc:930] 7 TS(s) registered with all masters
I20240502 20:45:30.454020 22709 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 1c52358e8e2f4ffe992cafe0f1a95a77 to finish bootstrapping
I20240502 20:45:30.463454 22709 ts_itest-base.cc:233] Waiting for 0 tablets on tserver f926d9d9d2314c2b90437934356f66d4 to finish bootstrapping
I20240502 20:45:30.468433 22709 ts_itest-base.cc:233] Waiting for 1 tablets on tserver a984b4b59cb6429290bc907a860d50f8 to finish bootstrapping
I20240502 20:45:30.473109 22709 ts_itest-base.cc:233] Waiting for 0 tablets on tserver bb697f05a8394524a411ece19fe608dc to finish bootstrapping
I20240502 20:45:30.478117 22709 ts_itest-base.cc:233] Waiting for 0 tablets on tserver eef5daac82024c449fc8b0dc82bd428c to finish bootstrapping
I20240502 20:45:30.482756 22709 ts_itest-base.cc:233] Waiting for 0 tablets on tserver f6dd93d055454bc5965b407745c020ff to finish bootstrapping
I20240502 20:45:30.487494 22709 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 85b4f683f6904c35b3c0e91799295bb3 to finish bootstrapping
I20240502 20:45:30.492205 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 24897
I20240502 20:45:30.497849 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 25028
I20240502 20:45:30.502676 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 25159
I20240502 20:45:30.507544 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 25290
I20240502 20:45:30.511893 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 25422
I20240502 20:45:34.267439 24573 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46840:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:45:34.279129 24828 tablet_service.cc:1467] Processing CreateTablet for tablet 0b736003ce014dbe8e11235e0d8e0872 (DEFAULT_TABLE table=rebalance_test_table_0 [id=f8e7a465361144c9a131d92402ce77fb]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:34.279129 24696 tablet_service.cc:1467] Processing CreateTablet for tablet d661eaefcd884d1b947002588ec335ac (DEFAULT_TABLE table=rebalance_test_table_0 [id=f8e7a465361144c9a131d92402ce77fb]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:34.279129 24695 tablet_service.cc:1467] Processing CreateTablet for tablet 77f37012797e41cb9aafd5ab00d049ab (DEFAULT_TABLE table=rebalance_test_table_0 [id=f8e7a465361144c9a131d92402ce77fb]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:34.280122 24828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b736003ce014dbe8e11235e0d8e0872. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:34.280125 24696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d661eaefcd884d1b947002588ec335ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:34.280453 24695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77f37012797e41cb9aafd5ab00d049ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:34.285534 25588 tablet_bootstrap.cc:492] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77: Bootstrap starting.
I20240502 20:45:34.286662 25587 tablet_bootstrap.cc:492] T 0b736003ce014dbe8e11235e0d8e0872 P f926d9d9d2314c2b90437934356f66d4: Bootstrap starting.
I20240502 20:45:34.287345 25588 tablet_bootstrap.cc:654] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:34.287964 25588 log.cc:824] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77: Log is configured to *not* fsync() on all Append() calls
I20240502 20:45:34.288280 25587 tablet_bootstrap.cc:654] T 0b736003ce014dbe8e11235e0d8e0872 P f926d9d9d2314c2b90437934356f66d4: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:34.288892 25587 log.cc:824] T 0b736003ce014dbe8e11235e0d8e0872 P f926d9d9d2314c2b90437934356f66d4: Log is configured to *not* fsync() on all Append() calls
I20240502 20:45:34.289484 25588 tablet_bootstrap.cc:492] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77: No bootstrap required, opened a new log
I20240502 20:45:34.289708 25588 ts_tablet_manager.cc:1367] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20240502 20:45:34.290221 25587 tablet_bootstrap.cc:492] T 0b736003ce014dbe8e11235e0d8e0872 P f926d9d9d2314c2b90437934356f66d4: No bootstrap required, opened a new log
I20240502 20:45:34.290372 25587 ts_tablet_manager.cc:1367] T 0b736003ce014dbe8e11235e0d8e0872 P f926d9d9d2314c2b90437934356f66d4: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 20:45:34.292553 25588 raft_consensus.cc:348] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:34.292732 25588 raft_consensus.cc:374] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:34.292766 25588 raft_consensus.cc:729] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c52358e8e2f4ffe992cafe0f1a95a77, State: Initialized, Role: FOLLOWER
I20240502 20:45:34.293061 25587 raft_consensus.cc:348] T 0b736003ce014dbe8e11235e0d8e0872 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:34.293159 25588 consensus_queue.cc:260] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:34.293208 25587 raft_consensus.cc:374] T 0b736003ce014dbe8e11235e0d8e0872 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:34.293267 25587 raft_consensus.cc:729] T 0b736003ce014dbe8e11235e0d8e0872 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f926d9d9d2314c2b90437934356f66d4, State: Initialized, Role: FOLLOWER
I20240502 20:45:34.293305 25588 raft_consensus.cc:388] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 20:45:34.293365 25588 raft_consensus.cc:482] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 20:45:34.293509 25588 raft_consensus.cc:3037] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:34.293642 25587 consensus_queue.cc:260] T 0b736003ce014dbe8e11235e0d8e0872 P f926d9d9d2314c2b90437934356f66d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:34.293752 25587 raft_consensus.cc:388] T 0b736003ce014dbe8e11235e0d8e0872 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 20:45:34.293807 25587 raft_consensus.cc:482] T 0b736003ce014dbe8e11235e0d8e0872 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 20:45:34.293903 25587 raft_consensus.cc:3037] T 0b736003ce014dbe8e11235e0d8e0872 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:34.294260 25588 raft_consensus.cc:504] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:34.294592 25588 leader_election.cc:302] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c52358e8e2f4ffe992cafe0f1a95a77; no voters:
I20240502 20:45:34.294627 25587 raft_consensus.cc:504] T 0b736003ce014dbe8e11235e0d8e0872 P f926d9d9d2314c2b90437934356f66d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:34.294847 25588 leader_election.cc:288] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 20:45:34.294885 25587 leader_election.cc:302] T 0b736003ce014dbe8e11235e0d8e0872 P f926d9d9d2314c2b90437934356f66d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f926d9d9d2314c2b90437934356f66d4; no voters:
I20240502 20:45:34.294917 25591 raft_consensus.cc:2781] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:45:34.295073 25591 raft_consensus.cc:686] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Becoming Leader. State: Replica: 1c52358e8e2f4ffe992cafe0f1a95a77, State: Running, Role: LEADER
I20240502 20:45:34.295157 25587 leader_election.cc:288] T 0b736003ce014dbe8e11235e0d8e0872 P f926d9d9d2314c2b90437934356f66d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 20:45:34.295351 25591 consensus_queue.cc:237] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:34.295652 25588 ts_tablet_manager.cc:1398] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240502 20:45:34.295773 25587 ts_tablet_manager.cc:1398] T 0b736003ce014dbe8e11235e0d8e0872 P f926d9d9d2314c2b90437934356f66d4: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20240502 20:45:34.295980 25588 tablet_bootstrap.cc:492] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77: Bootstrap starting.
I20240502 20:45:34.296808 25588 tablet_bootstrap.cc:654] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:34.297869 25588 tablet_bootstrap.cc:492] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77: No bootstrap required, opened a new log
I20240502 20:45:34.297895 25594 raft_consensus.cc:2781] T 0b736003ce014dbe8e11235e0d8e0872 P f926d9d9d2314c2b90437934356f66d4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:45:34.297948 25588 ts_tablet_manager.cc:1367] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:45:34.298072 25594 raft_consensus.cc:686] T 0b736003ce014dbe8e11235e0d8e0872 P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Becoming Leader. State: Replica: f926d9d9d2314c2b90437934356f66d4, State: Running, Role: LEADER
I20240502 20:45:34.298310 25594 consensus_queue.cc:237] T 0b736003ce014dbe8e11235e0d8e0872 P f926d9d9d2314c2b90437934356f66d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:34.298501 25588 raft_consensus.cc:348] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:34.298588 25588 raft_consensus.cc:374] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:34.298616 25588 raft_consensus.cc:729] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c52358e8e2f4ffe992cafe0f1a95a77, State: Initialized, Role: FOLLOWER
I20240502 20:45:34.298738 25588 consensus_queue.cc:260] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:34.299377 24573 catalog_manager.cc:5484] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65). New cstate: current_term: 1 leader_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } }
I20240502 20:45:34.300416 25588 raft_consensus.cc:388] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 20:45:34.300472 25588 raft_consensus.cc:482] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 20:45:34.300518 25588 raft_consensus.cc:3037] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:34.300623 24572 catalog_manager.cc:5484] T 0b736003ce014dbe8e11235e0d8e0872 P f926d9d9d2314c2b90437934356f66d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f926d9d9d2314c2b90437934356f66d4 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "f926d9d9d2314c2b90437934356f66d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } }
I20240502 20:45:34.301908 25588 raft_consensus.cc:504] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:34.302033 25588 leader_election.cc:302] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c52358e8e2f4ffe992cafe0f1a95a77; no voters:
I20240502 20:45:34.302116 25588 leader_election.cc:288] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 20:45:34.302233 25593 raft_consensus.cc:2781] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:45:34.302301 25588 ts_tablet_manager.cc:1398] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:45:34.302317 25593 raft_consensus.cc:686] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Becoming Leader. State: Replica: 1c52358e8e2f4ffe992cafe0f1a95a77, State: Running, Role: LEADER
I20240502 20:45:34.302414 25593 consensus_queue.cc:237] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:34.303970 24573 catalog_manager.cc:5484] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65). New cstate: current_term: 1 leader_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } }
W20240502 20:45:34.413381 25603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:34.413583 25603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:34.416498 25603 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
INSERT report
rows total: 2000
time total: 60.5547 ms
time per row: 0.0302773 ms
I20240502 20:45:34.515915 24572 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46840:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:45:34.520826 24827 tablet_service.cc:1467] Processing CreateTablet for tablet f258bd1f1bd14f99bed69e112cb09358 (DEFAULT_TABLE table=rebalance_test_table_1 [id=2214dd294ae048a4926821c50ed5e793]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:34.520826 24828 tablet_service.cc:1467] Processing CreateTablet for tablet 61dc146b84774046980dc8fdbf8e632f (DEFAULT_TABLE table=rebalance_test_table_1 [id=2214dd294ae048a4926821c50ed5e793]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:34.520944 24695 tablet_service.cc:1467] Processing CreateTablet for tablet a95565010b00430d977602e0a485a1dd (DEFAULT_TABLE table=rebalance_test_table_1 [id=2214dd294ae048a4926821c50ed5e793]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:34.521184 24828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61dc146b84774046980dc8fdbf8e632f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:34.521270 24695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a95565010b00430d977602e0a485a1dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:34.521489 24827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f258bd1f1bd14f99bed69e112cb09358. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:34.522950 25588 tablet_bootstrap.cc:492] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77: Bootstrap starting.
I20240502 20:45:34.523298 25587 tablet_bootstrap.cc:492] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4: Bootstrap starting.
I20240502 20:45:34.523636 25588 tablet_bootstrap.cc:654] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:34.523929 25587 tablet_bootstrap.cc:654] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:34.524603 25588 tablet_bootstrap.cc:492] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77: No bootstrap required, opened a new log
I20240502 20:45:34.524684 25588 ts_tablet_manager.cc:1367] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:45:34.524926 25587 tablet_bootstrap.cc:492] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4: No bootstrap required, opened a new log
I20240502 20:45:34.525018 25587 ts_tablet_manager.cc:1367] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:45:34.525053 25588 raft_consensus.cc:348] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:34.525141 25588 raft_consensus.cc:374] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:34.525171 25588 raft_consensus.cc:729] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c52358e8e2f4ffe992cafe0f1a95a77, State: Initialized, Role: FOLLOWER
I20240502 20:45:34.525280 25588 consensus_queue.cc:260] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:34.525369 25588 raft_consensus.cc:388] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 20:45:34.525405 25588 raft_consensus.cc:482] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 20:45:34.525447 25588 raft_consensus.cc:3037] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:34.525511 25587 raft_consensus.cc:348] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:34.525595 25587 raft_consensus.cc:374] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:34.525631 25587 raft_consensus.cc:729] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f926d9d9d2314c2b90437934356f66d4, State: Initialized, Role: FOLLOWER
I20240502 20:45:34.525743 25587 consensus_queue.cc:260] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:34.525823 25587 raft_consensus.cc:388] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 20:45:34.525861 25587 raft_consensus.cc:482] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 20:45:34.525905 25587 raft_consensus.cc:3037] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:34.526230 25588 raft_consensus.cc:504] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:34.526342 25588 leader_election.cc:302] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c52358e8e2f4ffe992cafe0f1a95a77; no voters:
I20240502 20:45:34.526495 25588 leader_election.cc:288] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 20:45:34.526551 25587 raft_consensus.cc:504] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:34.526588 25593 raft_consensus.cc:2781] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:45:34.526671 25593 raft_consensus.cc:686] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Becoming Leader. State: Replica: 1c52358e8e2f4ffe992cafe0f1a95a77, State: Running, Role: LEADER
I20240502 20:45:34.526674 25587 leader_election.cc:302] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f926d9d9d2314c2b90437934356f66d4; no voters:
I20240502 20:45:34.526753 25587 leader_election.cc:288] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 20:45:34.526774 25593 consensus_queue.cc:237] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:34.526799 25598 raft_consensus.cc:2781] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:45:34.526909 25598 raft_consensus.cc:686] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Becoming Leader. State: Replica: f926d9d9d2314c2b90437934356f66d4, State: Running, Role: LEADER
I20240502 20:45:34.527024 25587 ts_tablet_manager.cc:1398] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240502 20:45:34.527014 25598 consensus_queue.cc:237] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:34.527184 25587 tablet_bootstrap.cc:492] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4: Bootstrap starting.
I20240502 20:45:34.527526 25588 ts_tablet_manager.cc:1398] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:45:34.527904 25587 tablet_bootstrap.cc:654] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:34.528100 24572 catalog_manager.cc:5484] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65). New cstate: current_term: 1 leader_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } }
I20240502 20:45:34.528462 24565 catalog_manager.cc:5484] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f926d9d9d2314c2b90437934356f66d4 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "f926d9d9d2314c2b90437934356f66d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } }
I20240502 20:45:34.529168 25587 tablet_bootstrap.cc:492] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4: No bootstrap required, opened a new log
I20240502 20:45:34.529229 25587 ts_tablet_manager.cc:1367] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:45:34.529510 25587 raft_consensus.cc:348] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:34.529574 25587 raft_consensus.cc:374] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:34.529599 25587 raft_consensus.cc:729] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f926d9d9d2314c2b90437934356f66d4, State: Initialized, Role: FOLLOWER
I20240502 20:45:34.529696 25587 consensus_queue.cc:260] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:34.529779 25587 raft_consensus.cc:388] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 20:45:34.529819 25587 raft_consensus.cc:482] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 20:45:34.529856 25587 raft_consensus.cc:3037] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:34.530612 25587 raft_consensus.cc:504] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:34.530709 25587 leader_election.cc:302] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f926d9d9d2314c2b90437934356f66d4; no voters:
I20240502 20:45:34.530778 25587 leader_election.cc:288] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 20:45:34.530831 25594 raft_consensus.cc:2781] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:45:34.530894 25594 raft_consensus.cc:686] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Becoming Leader. State: Replica: f926d9d9d2314c2b90437934356f66d4, State: Running, Role: LEADER
I20240502 20:45:34.530958 25587 ts_tablet_manager.cc:1398] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240502 20:45:34.530984 25594 consensus_queue.cc:237] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:34.532550 24572 catalog_manager.cc:5484] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f926d9d9d2314c2b90437934356f66d4 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "f926d9d9d2314c2b90437934356f66d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } }
W20240502 20:45:34.635483 25647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:34.635674 25647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:34.638540 25647 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
INSERT report
rows total: 2000
time total: 68.764 ms
time per row: 0.034382 ms
I20240502 20:45:34.743849 24565 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46840:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:45:34.748699 24695 tablet_service.cc:1467] Processing CreateTablet for tablet e106fd5f2bc44c9e96bc38e2fd0dec1f (DEFAULT_TABLE table=rebalance_test_table_2 [id=dbf6f06f0516420ab4d3fb51341aa469]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:34.748844 24827 tablet_service.cc:1467] Processing CreateTablet for tablet ee9e40ee38ca422a8ca040a4a49b2ea7 (DEFAULT_TABLE table=rebalance_test_table_2 [id=dbf6f06f0516420ab4d3fb51341aa469]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:34.748998 24695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e106fd5f2bc44c9e96bc38e2fd0dec1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:34.749090 24827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee9e40ee38ca422a8ca040a4a49b2ea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:34.750994 25587 tablet_bootstrap.cc:492] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4: Bootstrap starting.
I20240502 20:45:34.750991 25588 tablet_bootstrap.cc:492] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77: Bootstrap starting.
I20240502 20:45:34.751185 24696 tablet_service.cc:1467] Processing CreateTablet for tablet e35840461c9f4b4da8921cb04d283a46 (DEFAULT_TABLE table=rebalance_test_table_2 [id=dbf6f06f0516420ab4d3fb51341aa469]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:34.751366 24696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e35840461c9f4b4da8921cb04d283a46. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:34.751832 25587 tablet_bootstrap.cc:654] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:34.751832 25588 tablet_bootstrap.cc:654] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:34.752950 25588 tablet_bootstrap.cc:492] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77: No bootstrap required, opened a new log
I20240502 20:45:34.753029 25588 ts_tablet_manager.cc:1367] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:45:34.753254 25587 tablet_bootstrap.cc:492] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4: No bootstrap required, opened a new log
I20240502 20:45:34.753340 25587 ts_tablet_manager.cc:1367] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 20:45:34.753458 25588 raft_consensus.cc:348] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:34.753564 25588 raft_consensus.cc:374] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:34.753603 25588 raft_consensus.cc:729] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c52358e8e2f4ffe992cafe0f1a95a77, State: Initialized, Role: FOLLOWER
I20240502 20:45:34.753724 25588 consensus_queue.cc:260] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:34.753759 25587 raft_consensus.cc:348] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:34.753815 25588 raft_consensus.cc:388] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 20:45:34.753844 25587 raft_consensus.cc:374] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:34.753849 25588 raft_consensus.cc:482] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 20:45:34.753881 25587 raft_consensus.cc:729] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f926d9d9d2314c2b90437934356f66d4, State: Initialized, Role: FOLLOWER
I20240502 20:45:34.753892 25588 raft_consensus.cc:3037] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:34.754005 25587 consensus_queue.cc:260] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:34.754091 25587 raft_consensus.cc:388] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 20:45:34.754128 25587 raft_consensus.cc:482] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 20:45:34.754170 25587 raft_consensus.cc:3037] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:34.754717 25588 raft_consensus.cc:504] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:34.754848 25588 leader_election.cc:302] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c52358e8e2f4ffe992cafe0f1a95a77; no voters:
I20240502 20:45:34.754961 25588 leader_election.cc:288] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 20:45:34.754984 25587 raft_consensus.cc:504] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:34.755044 25593 raft_consensus.cc:2781] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:45:34.755194 25588 ts_tablet_manager.cc:1398] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:45:34.755219 25593 raft_consensus.cc:686] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Becoming Leader. State: Replica: 1c52358e8e2f4ffe992cafe0f1a95a77, State: Running, Role: LEADER
I20240502 20:45:34.755450 25588 tablet_bootstrap.cc:492] T e35840461c9f4b4da8921cb04d283a46 P 1c52358e8e2f4ffe992cafe0f1a95a77: Bootstrap starting.
I20240502 20:45:34.755461 25587 leader_election.cc:302] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f926d9d9d2314c2b90437934356f66d4; no voters:
I20240502 20:45:34.755520 25593 consensus_queue.cc:237] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:34.755587 25594 raft_consensus.cc:2781] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:45:34.755650 25594 raft_consensus.cc:686] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Becoming Leader. State: Replica: f926d9d9d2314c2b90437934356f66d4, State: Running, Role: LEADER
I20240502 20:45:34.755744 25594 consensus_queue.cc:237] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:34.756347 25587 leader_election.cc:288] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 20:45:34.756760 25588 tablet_bootstrap.cc:654] T e35840461c9f4b4da8921cb04d283a46 P 1c52358e8e2f4ffe992cafe0f1a95a77: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:34.756851 24565 catalog_manager.cc:5484] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65). New cstate: current_term: 1 leader_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } }
I20240502 20:45:34.757005 24568 catalog_manager.cc:5484] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f926d9d9d2314c2b90437934356f66d4 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "f926d9d9d2314c2b90437934356f66d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } }
I20240502 20:45:34.757505 25587 ts_tablet_manager.cc:1398] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:45:34.758796 25588 tablet_bootstrap.cc:492] T e35840461c9f4b4da8921cb04d283a46 P 1c52358e8e2f4ffe992cafe0f1a95a77: No bootstrap required, opened a new log
I20240502 20:45:34.758881 25588 ts_tablet_manager.cc:1367] T e35840461c9f4b4da8921cb04d283a46 P 1c52358e8e2f4ffe992cafe0f1a95a77: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:45:34.759424 25588 raft_consensus.cc:348] T e35840461c9f4b4da8921cb04d283a46 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:34.759527 25588 raft_consensus.cc:374] T e35840461c9f4b4da8921cb04d283a46 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:34.759568 25588 raft_consensus.cc:729] T e35840461c9f4b4da8921cb04d283a46 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c52358e8e2f4ffe992cafe0f1a95a77, State: Initialized, Role: FOLLOWER
I20240502 20:45:34.759682 25588 consensus_queue.cc:260] T e35840461c9f4b4da8921cb04d283a46 P 1c52358e8e2f4ffe992cafe0f1a95a77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:34.759765 25588 raft_consensus.cc:388] T e35840461c9f4b4da8921cb04d283a46 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 20:45:34.759799 25588 raft_consensus.cc:482] T e35840461c9f4b4da8921cb04d283a46 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 20:45:34.759832 25588 raft_consensus.cc:3037] T e35840461c9f4b4da8921cb04d283a46 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:34.760576 25588 raft_consensus.cc:504] T e35840461c9f4b4da8921cb04d283a46 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:34.760702 25588 leader_election.cc:302] T e35840461c9f4b4da8921cb04d283a46 P 1c52358e8e2f4ffe992cafe0f1a95a77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c52358e8e2f4ffe992cafe0f1a95a77; no voters:
I20240502 20:45:34.760833 25593 raft_consensus.cc:2781] T e35840461c9f4b4da8921cb04d283a46 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:45:34.760906 25593 raft_consensus.cc:686] T e35840461c9f4b4da8921cb04d283a46 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Becoming Leader. State: Replica: 1c52358e8e2f4ffe992cafe0f1a95a77, State: Running, Role: LEADER
I20240502 20:45:34.761004 25593 consensus_queue.cc:237] T e35840461c9f4b4da8921cb04d283a46 P 1c52358e8e2f4ffe992cafe0f1a95a77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:34.761369 25588 leader_election.cc:288] T e35840461c9f4b4da8921cb04d283a46 P 1c52358e8e2f4ffe992cafe0f1a95a77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 20:45:34.761601 25588 ts_tablet_manager.cc:1398] T e35840461c9f4b4da8921cb04d283a46 P 1c52358e8e2f4ffe992cafe0f1a95a77: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:45:34.762308 24572 catalog_manager.cc:5484] T e35840461c9f4b4da8921cb04d283a46 P 1c52358e8e2f4ffe992cafe0f1a95a77 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65). New cstate: current_term: 1 leader_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } }
W20240502 20:45:34.868615 25679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:34.868813 25679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:34.871711 25679 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
INSERT report
rows total: 2000
time total: 76.812 ms
time per row: 0.038406 ms
I20240502 20:45:34.984814 24571 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46840:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:45:34.989563 24827 tablet_service.cc:1467] Processing CreateTablet for tablet 4a5012394f8e4ac19c07a1093b2dca14 (DEFAULT_TABLE table=rebalance_test_table_3 [id=8c45f90c8384402aaeaf9e17e4452dd9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:34.989627 24828 tablet_service.cc:1467] Processing CreateTablet for tablet 71e60972a8354e23adb03542b9935769 (DEFAULT_TABLE table=rebalance_test_table_3 [id=8c45f90c8384402aaeaf9e17e4452dd9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:34.989683 24696 tablet_service.cc:1467] Processing CreateTablet for tablet 7ed23be7cabc44b4ad4db0ee4f2e5392 (DEFAULT_TABLE table=rebalance_test_table_3 [id=8c45f90c8384402aaeaf9e17e4452dd9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:34.989851 24827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a5012394f8e4ac19c07a1093b2dca14. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:34.989928 24696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ed23be7cabc44b4ad4db0ee4f2e5392. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:34.989955 24828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71e60972a8354e23adb03542b9935769. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:34.991503 25587 tablet_bootstrap.cc:492] T 4a5012394f8e4ac19c07a1093b2dca14 P f926d9d9d2314c2b90437934356f66d4: Bootstrap starting.
I20240502 20:45:34.991848 25588 tablet_bootstrap.cc:492] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77: Bootstrap starting.
I20240502 20:45:34.992300 25587 tablet_bootstrap.cc:654] T 4a5012394f8e4ac19c07a1093b2dca14 P f926d9d9d2314c2b90437934356f66d4: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:34.992520 25588 tablet_bootstrap.cc:654] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:34.993252 25587 tablet_bootstrap.cc:492] T 4a5012394f8e4ac19c07a1093b2dca14 P f926d9d9d2314c2b90437934356f66d4: No bootstrap required, opened a new log
I20240502 20:45:34.993335 25587 ts_tablet_manager.cc:1367] T 4a5012394f8e4ac19c07a1093b2dca14 P f926d9d9d2314c2b90437934356f66d4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 20:45:34.993480 25588 tablet_bootstrap.cc:492] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77: No bootstrap required, opened a new log
I20240502 20:45:34.993556 25588 ts_tablet_manager.cc:1367] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:45:34.993865 25588 raft_consensus.cc:348] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:34.993942 25588 raft_consensus.cc:374] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:34.993961 25588 raft_consensus.cc:729] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c52358e8e2f4ffe992cafe0f1a95a77, State: Initialized, Role: FOLLOWER
I20240502 20:45:34.993940 25587 raft_consensus.cc:348] T 4a5012394f8e4ac19c07a1093b2dca14 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:34.994035 25587 raft_consensus.cc:374] T 4a5012394f8e4ac19c07a1093b2dca14 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:34.994066 25587 raft_consensus.cc:729] T 4a5012394f8e4ac19c07a1093b2dca14 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f926d9d9d2314c2b90437934356f66d4, State: Initialized, Role: FOLLOWER
I20240502 20:45:34.994060 25588 consensus_queue.cc:260] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:34.994132 25588 raft_consensus.cc:388] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 20:45:34.994170 25588 raft_consensus.cc:482] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 20:45:34.994210 25588 raft_consensus.cc:3037] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:34.994185 25587 consensus_queue.cc:260] T 4a5012394f8e4ac19c07a1093b2dca14 P f926d9d9d2314c2b90437934356f66d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:34.994248 25587 raft_consensus.cc:388] T 4a5012394f8e4ac19c07a1093b2dca14 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 20:45:34.994279 25587 raft_consensus.cc:482] T 4a5012394f8e4ac19c07a1093b2dca14 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 20:45:34.994316 25587 raft_consensus.cc:3037] T 4a5012394f8e4ac19c07a1093b2dca14 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:34.994895 25588 raft_consensus.cc:504] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:34.994923 25587 raft_consensus.cc:504] T 4a5012394f8e4ac19c07a1093b2dca14 P f926d9d9d2314c2b90437934356f66d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:34.995007 25588 leader_election.cc:302] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c52358e8e2f4ffe992cafe0f1a95a77; no voters:
I20240502 20:45:34.995031 25587 leader_election.cc:302] T 4a5012394f8e4ac19c07a1093b2dca14 P f926d9d9d2314c2b90437934356f66d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f926d9d9d2314c2b90437934356f66d4; no voters:
I20240502 20:45:34.995117 25588 leader_election.cc:288] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 20:45:34.995116 25587 leader_election.cc:288] T 4a5012394f8e4ac19c07a1093b2dca14 P f926d9d9d2314c2b90437934356f66d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 20:45:34.995172 25593 raft_consensus.cc:2781] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:45:34.995260 25593 raft_consensus.cc:686] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Becoming Leader. State: Replica: 1c52358e8e2f4ffe992cafe0f1a95a77, State: Running, Role: LEADER
I20240502 20:45:34.995329 25588 ts_tablet_manager.cc:1398] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:45:34.995358 25594 raft_consensus.cc:2781] T 4a5012394f8e4ac19c07a1093b2dca14 P f926d9d9d2314c2b90437934356f66d4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:45:34.995426 25594 raft_consensus.cc:686] T 4a5012394f8e4ac19c07a1093b2dca14 P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Becoming Leader. State: Replica: f926d9d9d2314c2b90437934356f66d4, State: Running, Role: LEADER
I20240502 20:45:34.995388 25593 consensus_queue.cc:237] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:34.995548 25594 consensus_queue.cc:237] T 4a5012394f8e4ac19c07a1093b2dca14 P f926d9d9d2314c2b90437934356f66d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:34.996273 25587 ts_tablet_manager.cc:1398] T 4a5012394f8e4ac19c07a1093b2dca14 P f926d9d9d2314c2b90437934356f66d4: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:45:34.996414 25587 tablet_bootstrap.cc:492] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4: Bootstrap starting.
I20240502 20:45:34.996609 24571 catalog_manager.cc:5484] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65). New cstate: current_term: 1 leader_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } }
I20240502 20:45:34.997138 24572 catalog_manager.cc:5484] T 4a5012394f8e4ac19c07a1093b2dca14 P f926d9d9d2314c2b90437934356f66d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f926d9d9d2314c2b90437934356f66d4 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "f926d9d9d2314c2b90437934356f66d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } }
I20240502 20:45:34.997342 25587 tablet_bootstrap.cc:654] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4: Neither blocks nor log segments found. Creating new log.
W20240502 20:45:34.998214 24764 tablet.cc:2369] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 20:45:34.999502 25587 tablet_bootstrap.cc:492] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4: No bootstrap required, opened a new log
I20240502 20:45:34.999562 25587 ts_tablet_manager.cc:1367] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:45:34.999933 25587 raft_consensus.cc:348] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:35.000034 25587 raft_consensus.cc:374] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:35.000059 25587 raft_consensus.cc:729] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f926d9d9d2314c2b90437934356f66d4, State: Initialized, Role: FOLLOWER
I20240502 20:45:35.000164 25587 consensus_queue.cc:260] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:35.000223 25587 raft_consensus.cc:388] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 20:45:35.000254 25587 raft_consensus.cc:482] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 20:45:35.000298 25587 raft_consensus.cc:3037] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:35.000867 25587 raft_consensus.cc:504] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:35.001000 25587 leader_election.cc:302] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f926d9d9d2314c2b90437934356f66d4; no voters:
I20240502 20:45:35.001094 25587 leader_election.cc:288] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 20:45:35.001139 25594 raft_consensus.cc:2781] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:45:35.001232 25594 raft_consensus.cc:686] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Becoming Leader. State: Replica: f926d9d9d2314c2b90437934356f66d4, State: Running, Role: LEADER
I20240502 20:45:35.001289 25587 ts_tablet_manager.cc:1398] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240502 20:45:35.001353 25594 consensus_queue.cc:237] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:35.002665 24572 catalog_manager.cc:5484] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f926d9d9d2314c2b90437934356f66d4 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "f926d9d9d2314c2b90437934356f66d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } }
W20240502 20:45:35.107239 25711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:35.107439 25711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:35.110356 25711 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
INSERT report
rows total: 2000
time total: 59.8526 ms
time per row: 0.0299263 ms
I20240502 20:45:35.206064 24572 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46840:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:45:35.210768 24696 tablet_service.cc:1467] Processing CreateTablet for tablet 3b6e517be2fe4c79bce310d8786554f4 (DEFAULT_TABLE table=rebalance_test_table_4 [id=493ba88faedf41dcb0c39446bb42569d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:35.210891 24828 tablet_service.cc:1467] Processing CreateTablet for tablet ca36c671f31b468fb2f68c0f70f4e928 (DEFAULT_TABLE table=rebalance_test_table_4 [id=493ba88faedf41dcb0c39446bb42569d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:35.210974 24695 tablet_service.cc:1467] Processing CreateTablet for tablet 21d2ddfb4dea4d32b3fac36a95e6a2b9 (DEFAULT_TABLE table=rebalance_test_table_4 [id=493ba88faedf41dcb0c39446bb42569d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:35.211079 24696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b6e517be2fe4c79bce310d8786554f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:35.211143 24828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca36c671f31b468fb2f68c0f70f4e928. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:35.211243 24695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21d2ddfb4dea4d32b3fac36a95e6a2b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:35.212829 25588 tablet_bootstrap.cc:492] T 21d2ddfb4dea4d32b3fac36a95e6a2b9 P 1c52358e8e2f4ffe992cafe0f1a95a77: Bootstrap starting.
I20240502 20:45:35.213060 25587 tablet_bootstrap.cc:492] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4: Bootstrap starting.
I20240502 20:45:35.213740 25588 tablet_bootstrap.cc:654] T 21d2ddfb4dea4d32b3fac36a95e6a2b9 P 1c52358e8e2f4ffe992cafe0f1a95a77: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:35.213793 25587 tablet_bootstrap.cc:654] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:35.214823 25588 tablet_bootstrap.cc:492] T 21d2ddfb4dea4d32b3fac36a95e6a2b9 P 1c52358e8e2f4ffe992cafe0f1a95a77: No bootstrap required, opened a new log
I20240502 20:45:35.214907 25588 ts_tablet_manager.cc:1367] T 21d2ddfb4dea4d32b3fac36a95e6a2b9 P 1c52358e8e2f4ffe992cafe0f1a95a77: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:45:35.215263 25588 raft_consensus.cc:348] T 21d2ddfb4dea4d32b3fac36a95e6a2b9 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:35.215342 25588 raft_consensus.cc:374] T 21d2ddfb4dea4d32b3fac36a95e6a2b9 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:35.215371 25588 raft_consensus.cc:729] T 21d2ddfb4dea4d32b3fac36a95e6a2b9 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c52358e8e2f4ffe992cafe0f1a95a77, State: Initialized, Role: FOLLOWER
I20240502 20:45:35.215488 25588 consensus_queue.cc:260] T 21d2ddfb4dea4d32b3fac36a95e6a2b9 P 1c52358e8e2f4ffe992cafe0f1a95a77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:35.215560 25588 raft_consensus.cc:388] T 21d2ddfb4dea4d32b3fac36a95e6a2b9 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 20:45:35.215598 25588 raft_consensus.cc:482] T 21d2ddfb4dea4d32b3fac36a95e6a2b9 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 20:45:35.215646 25588 raft_consensus.cc:3037] T 21d2ddfb4dea4d32b3fac36a95e6a2b9 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:35.216236 25588 raft_consensus.cc:504] T 21d2ddfb4dea4d32b3fac36a95e6a2b9 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:35.216352 25588 leader_election.cc:302] T 21d2ddfb4dea4d32b3fac36a95e6a2b9 P 1c52358e8e2f4ffe992cafe0f1a95a77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c52358e8e2f4ffe992cafe0f1a95a77; no voters:
I20240502 20:45:35.216437 25588 leader_election.cc:288] T 21d2ddfb4dea4d32b3fac36a95e6a2b9 P 1c52358e8e2f4ffe992cafe0f1a95a77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 20:45:35.216497 25593 raft_consensus.cc:2781] T 21d2ddfb4dea4d32b3fac36a95e6a2b9 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:45:35.216594 25593 raft_consensus.cc:686] T 21d2ddfb4dea4d32b3fac36a95e6a2b9 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Becoming Leader. State: Replica: 1c52358e8e2f4ffe992cafe0f1a95a77, State: Running, Role: LEADER
I20240502 20:45:35.216663 25588 ts_tablet_manager.cc:1398] T 21d2ddfb4dea4d32b3fac36a95e6a2b9 P 1c52358e8e2f4ffe992cafe0f1a95a77: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240502 20:45:35.216686 25593 consensus_queue.cc:237] T 21d2ddfb4dea4d32b3fac36a95e6a2b9 P 1c52358e8e2f4ffe992cafe0f1a95a77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:35.216799 25588 tablet_bootstrap.cc:492] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77: Bootstrap starting.
I20240502 20:45:35.217576 25588 tablet_bootstrap.cc:654] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:35.218153 24572 catalog_manager.cc:5484] T 21d2ddfb4dea4d32b3fac36a95e6a2b9 P 1c52358e8e2f4ffe992cafe0f1a95a77 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65). New cstate: current_term: 1 leader_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } }
I20240502 20:45:35.218523 25587 tablet_bootstrap.cc:492] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4: No bootstrap required, opened a new log
I20240502 20:45:35.218600 25587 ts_tablet_manager.cc:1367] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 20:45:35.218935 25587 raft_consensus.cc:348] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:35.219020 25587 raft_consensus.cc:374] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:35.219066 25587 raft_consensus.cc:729] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f926d9d9d2314c2b90437934356f66d4, State: Initialized, Role: FOLLOWER
I20240502 20:45:35.219182 25587 consensus_queue.cc:260] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:35.219273 25587 raft_consensus.cc:388] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 20:45:35.219300 25587 raft_consensus.cc:482] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 20:45:35.219341 25587 raft_consensus.cc:3037] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:35.220062 25587 raft_consensus.cc:504] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:35.220183 25587 leader_election.cc:302] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f926d9d9d2314c2b90437934356f66d4; no voters:
I20240502 20:45:35.220274 25587 leader_election.cc:288] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 20:45:35.220328 25595 raft_consensus.cc:2781] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:45:35.220429 25595 raft_consensus.cc:686] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Becoming Leader. State: Replica: f926d9d9d2314c2b90437934356f66d4, State: Running, Role: LEADER
I20240502 20:45:35.220504 25587 ts_tablet_manager.cc:1398] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240502 20:45:35.220532 25595 consensus_queue.cc:237] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:35.220927 25588 tablet_bootstrap.cc:492] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77: No bootstrap required, opened a new log
I20240502 20:45:35.221019 25588 ts_tablet_manager.cc:1367] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240502 20:45:35.221500 25588 raft_consensus.cc:348] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:35.221593 25588 raft_consensus.cc:374] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:35.221632 25588 raft_consensus.cc:729] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c52358e8e2f4ffe992cafe0f1a95a77, State: Initialized, Role: FOLLOWER
I20240502 20:45:35.221742 25588 consensus_queue.cc:260] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:35.221830 25588 raft_consensus.cc:388] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 20:45:35.221870 25588 raft_consensus.cc:482] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 20:45:35.221913 25588 raft_consensus.cc:3037] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:35.222205 24572 catalog_manager.cc:5484] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f926d9d9d2314c2b90437934356f66d4 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "f926d9d9d2314c2b90437934356f66d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } }
I20240502 20:45:35.222815 25588 raft_consensus.cc:504] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:35.222921 25588 leader_election.cc:302] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c52358e8e2f4ffe992cafe0f1a95a77; no voters:
I20240502 20:45:35.223019 25588 leader_election.cc:288] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 20:45:35.223076 25592 raft_consensus.cc:2781] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:45:35.223166 25592 raft_consensus.cc:686] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Becoming Leader. State: Replica: 1c52358e8e2f4ffe992cafe0f1a95a77, State: Running, Role: LEADER
I20240502 20:45:35.223255 25588 ts_tablet_manager.cc:1398] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20240502 20:45:35.223284 25592 consensus_queue.cc:237] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:35.224624 24572 catalog_manager.cc:5484] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65). New cstate: current_term: 1 leader_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } }
W20240502 20:45:35.332278 25746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:35.332465 25746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:35.335373 25746 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
INSERT report
rows total: 2000
time total: 70.278 ms
time per row: 0.035139 ms
I20240502 20:45:35.441639 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.67:44809
--local_ip_for_outbound_sockets=127.22.45.67
--tserver_master_addrs=127.22.45.126:34337
--webserver_port=44427
--webserver_interface=127.22.45.67
--builtin_ntp_servers=127.22.45.84:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 20:45:35.541442 25771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:35.541616 25771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:35.541654 25771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:35.544507 25771 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:45:35.544557 25771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:35.544620 25771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.67
I20240502 20:45:35.548009 25771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.67:44809
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.22.45.67
--webserver_port=44427
--tserver_master_addrs=127.22.45.126:34337
--never_fsync=true
--heap_profile_path=/tmp/kudu.25771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.67
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:35.548722 25771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:35.549718 25771 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:35.557348 25777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:35.557749 25776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:35.558050 25779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:35.558526 25771 server_base.cc:1034] running on GCE node
I20240502 20:45:35.558811 25771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:35.559265 25771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:35.560412 25771 hybrid_clock.cc:648] HybridClock initialized: now 1714682735560395 us; error 29 us; skew 500 ppm
I20240502 20:45:35.565259 25771 webserver.cc:460] Webserver started at http://127.22.45.67:44427/ using document root <none> and password file <none>
I20240502 20:45:35.565760 25771 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:35.565829 25771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:35.568936 25771 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:45:35.570744 25785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:35.571679 25771 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20240502 20:45:35.571774 25771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "a984b4b59cb6429290bc907a860d50f8"
format_stamp: "Formatted at 2024-05-02 20:45:29 on dist-test-slave-g7g5"
I20240502 20:45:35.572127 25771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:35.601212 25771 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:35.601742 25771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:35.601878 25771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:35.602334 25771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:35.603318 25792 ts_tablet_manager.cc:516] Loading tablet metadata (0/1 complete)
I20240502 20:45:35.606055 25771 ts_tablet_manager.cc:559] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20240502 20:45:35.606106 25771 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20240502 20:45:35.606163 25771 ts_tablet_manager.cc:574] Registering tablets (0/1 complete)
I20240502 20:45:35.607548 25771 ts_tablet_manager.cc:590] Registered 1 tablets
I20240502 20:45:35.607590 25771 ts_tablet_manager.cc:569] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20240502 20:45:35.607805 25792 tablet_bootstrap.cc:492] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8: Bootstrap starting.
I20240502 20:45:35.616976 25771 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.67:44809
I20240502 20:45:35.617019 25900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.67:44809 every 8 connection(s)
I20240502 20:45:35.617920 25771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 20:45:35.620306 25792 log.cc:824] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8: Log is configured to *not* fsync() on all Append() calls
I20240502 20:45:35.622821 25792 tablet_bootstrap.cc:492] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:45:35.623136 25792 tablet_bootstrap.cc:492] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8: Bootstrap complete.
I20240502 20:45:35.624243 25792 ts_tablet_manager.cc:1367] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.011s
I20240502 20:45:35.626272 25901 heartbeater.cc:347] Connected to a master server at 127.22.45.126:34337
I20240502 20:45:35.626504 25901 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:35.626776 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 25771
I20240502 20:45:35.626928 25901 heartbeater.cc:511] Master 127.22.45.126:34337 requested a full tablet report, sending...
I20240502 20:45:35.626843 25792 raft_consensus.cc:348] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a984b4b59cb6429290bc907a860d50f8" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44809 } }
I20240502 20:45:35.627208 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.68:39501
--local_ip_for_outbound_sockets=127.22.45.68
--tserver_master_addrs=127.22.45.126:34337
--webserver_port=41093
--webserver_interface=127.22.45.68
--builtin_ntp_servers=127.22.45.84:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:45:35.627218 25792 raft_consensus.cc:729] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a984b4b59cb6429290bc907a860d50f8, State: Initialized, Role: FOLLOWER
I20240502 20:45:35.627671 25792 consensus_queue.cc:260] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a984b4b59cb6429290bc907a860d50f8" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44809 } }
I20240502 20:45:35.627820 25792 raft_consensus.cc:388] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 20:45:35.627903 25792 raft_consensus.cc:482] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 20:45:35.628012 25792 raft_consensus.cc:3037] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:45:35.629415 24567 ts_manager.cc:194] Re-registered known tserver with Master: a984b4b59cb6429290bc907a860d50f8 (127.22.45.67:44809)
I20240502 20:45:35.629791 25792 raft_consensus.cc:504] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a984b4b59cb6429290bc907a860d50f8" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44809 } }
I20240502 20:45:35.630127 25792 leader_election.cc:302] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a984b4b59cb6429290bc907a860d50f8; no voters:
I20240502 20:45:35.630398 25792 leader_election.cc:288] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20240502 20:45:35.630504 25906 raft_consensus.cc:2781] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:45:35.630535 24567 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.67:55483
I20240502 20:45:35.630676 25906 raft_consensus.cc:686] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8 [term 2 LEADER]: Becoming Leader. State: Replica: a984b4b59cb6429290bc907a860d50f8, State: Running, Role: LEADER
I20240502 20:45:35.630959 25906 consensus_queue.cc:237] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a984b4b59cb6429290bc907a860d50f8" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44809 } }
I20240502 20:45:35.632561 25901 heartbeater.cc:503] Master 127.22.45.126:34337 was elected leader, sending a full tablet report...
I20240502 20:45:35.633445 24567 catalog_manager.cc:5484] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "a984b4b59cb6429290bc907a860d50f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a984b4b59cb6429290bc907a860d50f8" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44809 } } }
I20240502 20:45:35.635388 25792 ts_tablet_manager.cc:1398] T dec4ab92da444ee28d3c9ccaa498cb5e P a984b4b59cb6429290bc907a860d50f8: Time spent starting tablet: real 0.011s user 0.009s sys 0.000s
W20240502 20:45:35.728839 25905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:35.729017 25905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:35.729074 25905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:35.732053 25905 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:45:35.732111 25905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:35.732190 25905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.68
I20240502 20:45:35.735680 25905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.68:39501
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.22.45.68
--webserver_port=41093
--tserver_master_addrs=127.22.45.126:34337
--never_fsync=true
--heap_profile_path=/tmp/kudu.25905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.68
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:35.736380 25905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:35.737327 25905 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:35.744536 25915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:35.744824 25918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:35.745210 25916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:35.745663 25905 server_base.cc:1034] running on GCE node
I20240502 20:45:35.745944 25905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:35.746374 25905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:35.747509 25905 hybrid_clock.cc:648] HybridClock initialized: now 1714682735747493 us; error 28 us; skew 500 ppm
I20240502 20:45:35.749043 25905 webserver.cc:460] Webserver started at http://127.22.45.68:41093/ using document root <none> and password file <none>
I20240502 20:45:35.749511 25905 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:35.749577 25905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:35.752610 25905 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20240502 20:45:35.754256 25925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:35.755187 25905 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:45:35.755282 25905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "bb697f05a8394524a411ece19fe608dc"
format_stamp: "Formatted at 2024-05-02 20:45:29 on dist-test-slave-g7g5"
I20240502 20:45:35.755628 25905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:35.771313 25905 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:35.771786 25905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:35.771898 25905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:35.772313 25905 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:35.773116 25905 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:35.773159 25905 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:35.773193 25905 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:35.773213 25905 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:35.780732 25905 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.68:39501
I20240502 20:45:35.780750 26039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.68:39501 every 8 connection(s)
I20240502 20:45:35.781579 25905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 20:45:35.783578 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 25905
I20240502 20:45:35.784011 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.69:37677
--local_ip_for_outbound_sockets=127.22.45.69
--tserver_master_addrs=127.22.45.126:34337
--webserver_port=37115
--webserver_interface=127.22.45.69
--builtin_ntp_servers=127.22.45.84:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:45:35.790130 26040 heartbeater.cc:347] Connected to a master server at 127.22.45.126:34337
I20240502 20:45:35.790315 26040 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:35.790724 26040 heartbeater.cc:511] Master 127.22.45.126:34337 requested a full tablet report, sending...
I20240502 20:45:35.791605 24567 ts_manager.cc:194] Re-registered known tserver with Master: bb697f05a8394524a411ece19fe608dc (127.22.45.68:39501)
I20240502 20:45:35.792330 24567 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.68:32957
W20240502 20:45:35.882673 26043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:35.882836 26043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:35.882879 26043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:35.885772 26043 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:45:35.885825 26043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:35.885886 26043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.69
I20240502 20:45:35.889312 26043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.69:37677
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.22.45.69
--webserver_port=37115
--tserver_master_addrs=127.22.45.126:34337
--never_fsync=true
--heap_profile_path=/tmp/kudu.26043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.69
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:35.889993 26043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:35.890931 26043 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:35.897709 26051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:35.898061 26048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:35.898422 26049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:35.898883 26043 server_base.cc:1034] running on GCE node
I20240502 20:45:35.899179 26043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:35.899641 26043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:35.900789 26043 hybrid_clock.cc:648] HybridClock initialized: now 1714682735900761 us; error 40 us; skew 500 ppm
I20240502 20:45:35.902276 26043 webserver.cc:460] Webserver started at http://127.22.45.69:37115/ using document root <none> and password file <none>
I20240502 20:45:35.902768 26043 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:35.902837 26043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:35.905692 26043 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:45:35.907421 26057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:35.908270 26043 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 20:45:35.908365 26043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "eef5daac82024c449fc8b0dc82bd428c"
format_stamp: "Formatted at 2024-05-02 20:45:29 on dist-test-slave-g7g5"
I20240502 20:45:35.908795 26043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:35.924959 26043 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:35.925451 26043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:35.925587 26043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:35.926030 26043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:35.926816 26043 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:35.926862 26043 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:35.926908 26043 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:35.926934 26043 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:35.934643 26043 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.69:37677
I20240502 20:45:35.934674 26171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.69:37677 every 8 connection(s)
I20240502 20:45:35.935489 26043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 20:45:35.939198 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 26043
I20240502 20:45:35.939525 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.70:36617
--local_ip_for_outbound_sockets=127.22.45.70
--tserver_master_addrs=127.22.45.126:34337
--webserver_port=33401
--webserver_interface=127.22.45.70
--builtin_ntp_servers=127.22.45.84:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:45:35.943145 26172 heartbeater.cc:347] Connected to a master server at 127.22.45.126:34337
I20240502 20:45:35.943331 26172 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:35.943701 26172 heartbeater.cc:511] Master 127.22.45.126:34337 requested a full tablet report, sending...
I20240502 20:45:35.944541 24567 ts_manager.cc:194] Re-registered known tserver with Master: eef5daac82024c449fc8b0dc82bd428c (127.22.45.69:37677)
I20240502 20:45:35.945058 24567 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.69:57163
W20240502 20:45:36.041124 26175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:36.041296 26175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:36.041339 26175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:36.044298 26175 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:45:36.044351 26175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:36.044438 26175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.70
I20240502 20:45:36.047812 26175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.70:36617
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.22.45.70
--webserver_port=33401
--tserver_master_addrs=127.22.45.126:34337
--never_fsync=true
--heap_profile_path=/tmp/kudu.26175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.70
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:36.048575 26175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:36.049523 26175 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:36.056560 26183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:36.056908 26180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:36.057258 26181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:36.057683 26175 server_base.cc:1034] running on GCE node
I20240502 20:45:36.057968 26175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:36.058419 26175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:36.059584 26175 hybrid_clock.cc:648] HybridClock initialized: now 1714682736059568 us; error 28 us; skew 500 ppm
I20240502 20:45:36.061141 26175 webserver.cc:460] Webserver started at http://127.22.45.70:33401/ using document root <none> and password file <none>
I20240502 20:45:36.061611 26175 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:36.061676 26175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:36.064594 26175 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20240502 20:45:36.066201 26189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:36.067091 26175 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:45:36.067184 26175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "f6dd93d055454bc5965b407745c020ff"
format_stamp: "Formatted at 2024-05-02 20:45:30 on dist-test-slave-g7g5"
I20240502 20:45:36.067533 26175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:36.086827 26175 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:36.087334 26175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:36.087463 26175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:36.087901 26175 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:36.088660 26175 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:36.088704 26175 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:36.088749 26175 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:36.088781 26175 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:36.096659 26175 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.70:36617
I20240502 20:45:36.096689 26303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.70:36617 every 8 connection(s)
I20240502 20:45:36.097517 26175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 20:45:36.104283 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 26175
I20240502 20:45:36.104689 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.71:39125
--local_ip_for_outbound_sockets=127.22.45.71
--tserver_master_addrs=127.22.45.126:34337
--webserver_port=39827
--webserver_interface=127.22.45.71
--builtin_ntp_servers=127.22.45.84:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:45:36.105183 26304 heartbeater.cc:347] Connected to a master server at 127.22.45.126:34337
I20240502 20:45:36.105362 26304 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:36.105721 26304 heartbeater.cc:511] Master 127.22.45.126:34337 requested a full tablet report, sending...
I20240502 20:45:36.106619 24567 ts_manager.cc:194] Re-registered known tserver with Master: f6dd93d055454bc5965b407745c020ff (127.22.45.70:36617)
I20240502 20:45:36.107138 24567 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.70:58443
W20240502 20:45:36.204926 26307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:36.205104 26307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:36.205148 26307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:36.208142 26307 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:45:36.208196 26307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:36.208273 26307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.71
I20240502 20:45:36.211803 26307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.71:39125
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.22.45.71
--webserver_port=39827
--tserver_master_addrs=127.22.45.126:34337
--never_fsync=true
--heap_profile_path=/tmp/kudu.26307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.71
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:36.212524 26307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:36.213510 26307 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:36.220578 26312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:36.220870 26313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:36.221254 26315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:36.221657 26307 server_base.cc:1034] running on GCE node
I20240502 20:45:36.221959 26307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:36.222452 26307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:36.223626 26307 hybrid_clock.cc:648] HybridClock initialized: now 1714682736223620 us; error 24 us; skew 500 ppm
I20240502 20:45:36.225178 26307 webserver.cc:460] Webserver started at http://127.22.45.71:39827/ using document root <none> and password file <none>
I20240502 20:45:36.225647 26307 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:36.225713 26307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:36.229055 26307 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:45:36.230691 26321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:36.231570 26307 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:45:36.231662 26307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "85b4f683f6904c35b3c0e91799295bb3"
format_stamp: "Formatted at 2024-05-02 20:45:30 on dist-test-slave-g7g5"
I20240502 20:45:36.232017 26307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:36.249287 26307 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:36.249883 26307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:36.250027 26307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:36.250535 26307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:36.251420 26307 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:36.251469 26307 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:36.251511 26307 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:36.251536 26307 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:36.260522 26307 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.71:39125
I20240502 20:45:36.260556 26434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.71:39125 every 8 connection(s)
I20240502 20:45:36.261399 26307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 20:45:36.268551 26435 heartbeater.cc:347] Connected to a master server at 127.22.45.126:34337
I20240502 20:45:36.268729 26435 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:36.269099 26435 heartbeater.cc:511] Master 127.22.45.126:34337 requested a full tablet report, sending...
I20240502 20:45:36.269804 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 26307
I20240502 20:45:36.269924 24567 ts_manager.cc:194] Re-registered known tserver with Master: 85b4f683f6904c35b3c0e91799295bb3 (127.22.45.71:39125)
I20240502 20:45:36.270593 24567 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.71:44335
I20240502 20:45:36.443382 26369 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:36.444741 24696 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:45:36.446117 26105 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:36.449568 24828 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:45:36.452704 26237 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:36.453339 25973 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:36.464943 25834 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:36.560940 26369 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:36.561810 24696 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:45:36.564816 26105 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:36.572373 25834 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:36.574153 26237 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:36.574920 24828 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:45:36.579954 25973 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:36.624033 24848 consensus_queue.cc:237] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "f6dd93d055454bc5965b407745c020ff" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 36617 } }
W20240502 20:45:36.629182 24783 consensus_peers.cc:485] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4 -> Peer f6dd93d055454bc5965b407745c020ff (127.22.45.70:36617): Couldn't send request to peer f6dd93d055454bc5965b407745c020ff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 71e60972a8354e23adb03542b9935769. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:45:36.793969 26040 heartbeater.cc:503] Master 127.22.45.126:34337 was elected leader, sending a full tablet report...
I20240502 20:45:36.946642 26172 heartbeater.cc:503] Master 127.22.45.126:34337 was elected leader, sending a full tablet report...
I20240502 20:45:37.107313 26539 ts_tablet_manager.cc:902] T 71e60972a8354e23adb03542b9935769 P f6dd93d055454bc5965b407745c020ff: Initiating tablet copy from peer f926d9d9d2314c2b90437934356f66d4 (127.22.45.66:46041)
I20240502 20:45:37.107995 26539 tablet_copy_client.cc:308] T 71e60972a8354e23adb03542b9935769 P f6dd93d055454bc5965b407745c020ff: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.66:46041
I20240502 20:45:37.108772 26304 heartbeater.cc:503] Master 127.22.45.126:34337 was elected leader, sending a full tablet report...
I20240502 20:45:37.111891 24868 tablet_copy_service.cc:140] P f926d9d9d2314c2b90437934356f66d4: Received BeginTabletCopySession request for tablet 71e60972a8354e23adb03542b9935769 from peer f6dd93d055454bc5965b407745c020ff ({username='slave'} at 127.22.45.70:40711)
I20240502 20:45:37.112030 24868 tablet_copy_service.cc:161] P f926d9d9d2314c2b90437934356f66d4: Beginning new tablet copy session on tablet 71e60972a8354e23adb03542b9935769 from peer f6dd93d055454bc5965b407745c020ff at {username='slave'} at 127.22.45.70:40711: session id = f6dd93d055454bc5965b407745c020ff-71e60972a8354e23adb03542b9935769
I20240502 20:45:37.113286 24868 tablet_copy_source_session.cc:205] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:45:37.116245 26539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71e60972a8354e23adb03542b9935769. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:37.121335 26539 tablet_copy_client.cc:783] T 71e60972a8354e23adb03542b9935769 P f6dd93d055454bc5965b407745c020ff: tablet copy: Starting download of 0 data blocks...
I20240502 20:45:37.121517 26539 tablet_copy_client.cc:647] T 71e60972a8354e23adb03542b9935769 P f6dd93d055454bc5965b407745c020ff: tablet copy: Starting download of 1 WAL segments...
I20240502 20:45:37.123103 26539 tablet_copy_client.cc:523] T 71e60972a8354e23adb03542b9935769 P f6dd93d055454bc5965b407745c020ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:45:37.124375 26539 tablet_bootstrap.cc:492] T 71e60972a8354e23adb03542b9935769 P f6dd93d055454bc5965b407745c020ff: Bootstrap starting.
I20240502 20:45:37.128896 26539 log.cc:824] T 71e60972a8354e23adb03542b9935769 P f6dd93d055454bc5965b407745c020ff: Log is configured to *not* fsync() on all Append() calls
I20240502 20:45:37.145972 26539 tablet_bootstrap.cc:492] T 71e60972a8354e23adb03542b9935769 P f6dd93d055454bc5965b407745c020ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 20:45:37.146281 26539 tablet_bootstrap.cc:492] T 71e60972a8354e23adb03542b9935769 P f6dd93d055454bc5965b407745c020ff: Bootstrap complete.
I20240502 20:45:37.146531 26539 ts_tablet_manager.cc:1367] T 71e60972a8354e23adb03542b9935769 P f6dd93d055454bc5965b407745c020ff: Time spent bootstrapping tablet: real 0.022s user 0.016s sys 0.008s
I20240502 20:45:37.148231 26539 raft_consensus.cc:348] T 71e60972a8354e23adb03542b9935769 P f6dd93d055454bc5965b407745c020ff [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:37.148722 26539 raft_consensus.cc:729] T 71e60972a8354e23adb03542b9935769 P f6dd93d055454bc5965b407745c020ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6dd93d055454bc5965b407745c020ff, State: Initialized, Role: FOLLOWER
I20240502 20:45:37.148998 26539 consensus_queue.cc:260] T 71e60972a8354e23adb03542b9935769 P f6dd93d055454bc5965b407745c020ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "f6dd93d055454bc5965b407745c020ff" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 36617 } }
I20240502 20:45:37.149766 26539 ts_tablet_manager.cc:1398] T 71e60972a8354e23adb03542b9935769 P f6dd93d055454bc5965b407745c020ff: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20240502 20:45:37.150416 24868 tablet_copy_service.cc:342] P f926d9d9d2314c2b90437934356f66d4: Request end of tablet copy session f6dd93d055454bc5965b407745c020ff-71e60972a8354e23adb03542b9935769 received from {username='slave'} at 127.22.45.70:40711
I20240502 20:45:37.150519 24868 tablet_copy_service.cc:433] P f926d9d9d2314c2b90437934356f66d4: ending tablet copy session f6dd93d055454bc5965b407745c020ff-71e60972a8354e23adb03542b9935769 on tablet 71e60972a8354e23adb03542b9935769 with peer f6dd93d055454bc5965b407745c020ff
I20240502 20:45:37.272361 26435 heartbeater.cc:503] Master 127.22.45.126:34337 was elected leader, sending a full tablet report...
I20240502 20:45:37.593992 26257 raft_consensus.cc:1203] T 71e60972a8354e23adb03542b9935769 P f6dd93d055454bc5965b407745c020ff [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:45:37.595309 26547 raft_consensus.cc:2932] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f6dd93d055454bc5965b407745c020ff (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "f6dd93d055454bc5965b407745c020ff" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 36617 } } }
I20240502 20:45:37.595750 26257 raft_consensus.cc:2932] T 71e60972a8354e23adb03542b9935769 P f6dd93d055454bc5965b407745c020ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f6dd93d055454bc5965b407745c020ff (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "f6dd93d055454bc5965b407745c020ff" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 36617 } } }
I20240502 20:45:37.598068 24564 catalog_manager.cc:5484] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4 reported cstate change: config changed from index -1 to 4, VOTER f6dd93d055454bc5965b407745c020ff (127.22.45.70) added. New cstate: current_term: 1 leader_uuid: "f926d9d9d2314c2b90437934356f66d4" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "f6dd93d055454bc5965b407745c020ff" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 36617 } } }
I20240502 20:45:37.615029 24716 consensus_queue.cc:237] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "eef5daac82024c449fc8b0dc82bd428c" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 37677 } }
W20240502 20:45:37.620767 24651 consensus_peers.cc:485] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77 -> Peer eef5daac82024c449fc8b0dc82bd428c (127.22.45.69:37677): Couldn't send request to peer eef5daac82024c449fc8b0dc82bd428c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e106fd5f2bc44c9e96bc38e2fd0dec1f. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:45:38.122489 26575 ts_tablet_manager.cc:902] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c: Initiating tablet copy from peer 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65:36905)
I20240502 20:45:38.123216 26575 tablet_copy_client.cc:308] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.65:36905
I20240502 20:45:38.126780 24736 tablet_copy_service.cc:140] P 1c52358e8e2f4ffe992cafe0f1a95a77: Received BeginTabletCopySession request for tablet e106fd5f2bc44c9e96bc38e2fd0dec1f from peer eef5daac82024c449fc8b0dc82bd428c ({username='slave'} at 127.22.45.69:36663)
I20240502 20:45:38.126909 24736 tablet_copy_service.cc:161] P 1c52358e8e2f4ffe992cafe0f1a95a77: Beginning new tablet copy session on tablet e106fd5f2bc44c9e96bc38e2fd0dec1f from peer eef5daac82024c449fc8b0dc82bd428c at {username='slave'} at 127.22.45.69:36663: session id = eef5daac82024c449fc8b0dc82bd428c-e106fd5f2bc44c9e96bc38e2fd0dec1f
I20240502 20:45:38.127985 24736 tablet_copy_source_session.cc:205] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:45:38.131049 26575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e106fd5f2bc44c9e96bc38e2fd0dec1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:38.135988 26575 tablet_copy_client.cc:783] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c: tablet copy: Starting download of 0 data blocks...
I20240502 20:45:38.136145 26575 tablet_copy_client.cc:647] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c: tablet copy: Starting download of 1 WAL segments...
I20240502 20:45:38.137578 26575 tablet_copy_client.cc:523] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:45:38.138911 26575 tablet_bootstrap.cc:492] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c: Bootstrap starting.
I20240502 20:45:38.143548 26575 log.cc:824] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c: Log is configured to *not* fsync() on all Append() calls
I20240502 20:45:38.161696 26575 tablet_bootstrap.cc:492] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 20:45:38.162009 26575 tablet_bootstrap.cc:492] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c: Bootstrap complete.
I20240502 20:45:38.162245 26575 ts_tablet_manager.cc:1367] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c: Time spent bootstrapping tablet: real 0.023s user 0.020s sys 0.004s
I20240502 20:45:38.163920 26575 raft_consensus.cc:348] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:38.164413 26575 raft_consensus.cc:729] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eef5daac82024c449fc8b0dc82bd428c, State: Initialized, Role: FOLLOWER
I20240502 20:45:38.164671 26575 consensus_queue.cc:260] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "eef5daac82024c449fc8b0dc82bd428c" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 37677 } }
I20240502 20:45:38.165410 26575 ts_tablet_manager.cc:1398] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20240502 20:45:38.166069 24736 tablet_copy_service.cc:342] P 1c52358e8e2f4ffe992cafe0f1a95a77: Request end of tablet copy session eef5daac82024c449fc8b0dc82bd428c-e106fd5f2bc44c9e96bc38e2fd0dec1f received from {username='slave'} at 127.22.45.69:36663
I20240502 20:45:38.166149 24736 tablet_copy_service.cc:433] P 1c52358e8e2f4ffe992cafe0f1a95a77: ending tablet copy session eef5daac82024c449fc8b0dc82bd428c-e106fd5f2bc44c9e96bc38e2fd0dec1f on tablet e106fd5f2bc44c9e96bc38e2fd0dec1f with peer eef5daac82024c449fc8b0dc82bd428c
I20240502 20:45:38.707029 26125 raft_consensus.cc:1203] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:45:38.708390 26586 raft_consensus.cc:2932] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER eef5daac82024c449fc8b0dc82bd428c (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "eef5daac82024c449fc8b0dc82bd428c" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 37677 } } }
I20240502 20:45:38.708977 26125 raft_consensus.cc:2932] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER eef5daac82024c449fc8b0dc82bd428c (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "eef5daac82024c449fc8b0dc82bd428c" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 37677 } } }
I20240502 20:45:38.710184 26586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:58002 (request call id 0) took 1095 ms. Trace:
I20240502 20:45:38.710304 26586 rpcz_store.cc:276] 0502 20:45:37.614382 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 20:45:37.614432 (+ 50us) service_pool.cc:225] Handling call
0502 20:45:37.615771 (+ 1339us) log.cc:842] Serialized 257 byte log entry
0502 20:45:38.710166 (+1094395us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":44,"raft.queue_time_us":108,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":241,"thread_start_us":107,"threads_started":2,"wal-append.queue_time_us":130,"wal-append.run_cpu_time_us":330,"wal-append.run_wall_time_us":1000464}
I20240502 20:45:38.711571 24564 catalog_manager.cc:5484] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c reported cstate change: config changed from index -1 to 4, VOTER eef5daac82024c449fc8b0dc82bd428c (127.22.45.69) added. New cstate: current_term: 1 leader_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "eef5daac82024c449fc8b0dc82bd428c" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 37677 } } }
I20240502 20:45:38.728205 24848 consensus_queue.cc:237] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "bb697f05a8394524a411ece19fe608dc" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 39501 } }
W20240502 20:45:38.732353 24782 consensus_peers.cc:485] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4 -> Peer bb697f05a8394524a411ece19fe608dc (127.22.45.68:39501): Couldn't send request to peer bb697f05a8394524a411ece19fe608dc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ee9e40ee38ca422a8ca040a4a49b2ea7. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:45:39.157723 26613 ts_tablet_manager.cc:902] T ee9e40ee38ca422a8ca040a4a49b2ea7 P bb697f05a8394524a411ece19fe608dc: Initiating tablet copy from peer f926d9d9d2314c2b90437934356f66d4 (127.22.45.66:46041)
I20240502 20:45:39.158492 26613 tablet_copy_client.cc:308] T ee9e40ee38ca422a8ca040a4a49b2ea7 P bb697f05a8394524a411ece19fe608dc: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.66:46041
I20240502 20:45:39.161813 24868 tablet_copy_service.cc:140] P f926d9d9d2314c2b90437934356f66d4: Received BeginTabletCopySession request for tablet ee9e40ee38ca422a8ca040a4a49b2ea7 from peer bb697f05a8394524a411ece19fe608dc ({username='slave'} at 127.22.45.68:35097)
I20240502 20:45:39.161916 24868 tablet_copy_service.cc:161] P f926d9d9d2314c2b90437934356f66d4: Beginning new tablet copy session on tablet ee9e40ee38ca422a8ca040a4a49b2ea7 from peer bb697f05a8394524a411ece19fe608dc at {username='slave'} at 127.22.45.68:35097: session id = bb697f05a8394524a411ece19fe608dc-ee9e40ee38ca422a8ca040a4a49b2ea7
I20240502 20:45:39.162667 24868 tablet_copy_source_session.cc:205] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:45:39.165437 26613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee9e40ee38ca422a8ca040a4a49b2ea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:39.170397 26613 tablet_copy_client.cc:783] T ee9e40ee38ca422a8ca040a4a49b2ea7 P bb697f05a8394524a411ece19fe608dc: tablet copy: Starting download of 0 data blocks...
I20240502 20:45:39.170608 26613 tablet_copy_client.cc:647] T ee9e40ee38ca422a8ca040a4a49b2ea7 P bb697f05a8394524a411ece19fe608dc: tablet copy: Starting download of 1 WAL segments...
I20240502 20:45:39.171963 26613 tablet_copy_client.cc:523] T ee9e40ee38ca422a8ca040a4a49b2ea7 P bb697f05a8394524a411ece19fe608dc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:45:39.173293 26613 tablet_bootstrap.cc:492] T ee9e40ee38ca422a8ca040a4a49b2ea7 P bb697f05a8394524a411ece19fe608dc: Bootstrap starting.
I20240502 20:45:39.177865 26613 log.cc:824] T ee9e40ee38ca422a8ca040a4a49b2ea7 P bb697f05a8394524a411ece19fe608dc: Log is configured to *not* fsync() on all Append() calls
I20240502 20:45:39.195878 26613 tablet_bootstrap.cc:492] T ee9e40ee38ca422a8ca040a4a49b2ea7 P bb697f05a8394524a411ece19fe608dc: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 20:45:39.196184 26613 tablet_bootstrap.cc:492] T ee9e40ee38ca422a8ca040a4a49b2ea7 P bb697f05a8394524a411ece19fe608dc: Bootstrap complete.
I20240502 20:45:39.196420 26613 ts_tablet_manager.cc:1367] T ee9e40ee38ca422a8ca040a4a49b2ea7 P bb697f05a8394524a411ece19fe608dc: Time spent bootstrapping tablet: real 0.023s user 0.022s sys 0.000s
I20240502 20:45:39.198109 26613 raft_consensus.cc:348] T ee9e40ee38ca422a8ca040a4a49b2ea7 P bb697f05a8394524a411ece19fe608dc [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:39.198616 26613 raft_consensus.cc:729] T ee9e40ee38ca422a8ca040a4a49b2ea7 P bb697f05a8394524a411ece19fe608dc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb697f05a8394524a411ece19fe608dc, State: Initialized, Role: FOLLOWER
I20240502 20:45:39.198890 26613 consensus_queue.cc:260] T ee9e40ee38ca422a8ca040a4a49b2ea7 P bb697f05a8394524a411ece19fe608dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "bb697f05a8394524a411ece19fe608dc" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 39501 } }
I20240502 20:45:39.199626 26613 ts_tablet_manager.cc:1398] T ee9e40ee38ca422a8ca040a4a49b2ea7 P bb697f05a8394524a411ece19fe608dc: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
I20240502 20:45:39.200206 24868 tablet_copy_service.cc:342] P f926d9d9d2314c2b90437934356f66d4: Request end of tablet copy session bb697f05a8394524a411ece19fe608dc-ee9e40ee38ca422a8ca040a4a49b2ea7 received from {username='slave'} at 127.22.45.68:35097
I20240502 20:45:39.200286 24868 tablet_copy_service.cc:433] P f926d9d9d2314c2b90437934356f66d4: ending tablet copy session bb697f05a8394524a411ece19fe608dc-ee9e40ee38ca422a8ca040a4a49b2ea7 on tablet ee9e40ee38ca422a8ca040a4a49b2ea7 with peer bb697f05a8394524a411ece19fe608dc
I20240502 20:45:39.759351 25993 raft_consensus.cc:1203] T ee9e40ee38ca422a8ca040a4a49b2ea7 P bb697f05a8394524a411ece19fe608dc [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:45:39.760653 26622 raft_consensus.cc:2932] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER bb697f05a8394524a411ece19fe608dc (127.22.45.68) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "bb697f05a8394524a411ece19fe608dc" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 39501 } } }
I20240502 20:45:39.761226 25993 raft_consensus.cc:2932] T ee9e40ee38ca422a8ca040a4a49b2ea7 P bb697f05a8394524a411ece19fe608dc [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER bb697f05a8394524a411ece19fe608dc (127.22.45.68) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "bb697f05a8394524a411ece19fe608dc" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 39501 } } }
I20240502 20:45:39.761971 26622 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:44224 (request call id 0) took 1034 ms. Trace:
I20240502 20:45:39.762089 26622 rpcz_store.cc:276] 0502 20:45:38.727904 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 20:45:38.727955 (+ 51us) service_pool.cc:225] Handling call
0502 20:45:38.728580 (+ 625us) log.cc:842] Serialized 257 byte log entry
0502 20:45:39.761944 (+1033364us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":36,"raft.queue_time_us":35,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":135,"thread_start_us":64,"threads_started":1,"wal-append.queue_time_us":128,"wal-append.run_cpu_time_us":287,"wal-append.run_wall_time_us":1000346}
I20240502 20:45:39.763846 24564 catalog_manager.cc:5484] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4 reported cstate change: config changed from index -1 to 4, VOTER bb697f05a8394524a411ece19fe608dc (127.22.45.68) added. New cstate: current_term: 1 leader_uuid: "f926d9d9d2314c2b90437934356f66d4" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "bb697f05a8394524a411ece19fe608dc" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 39501 } } }
I20240502 20:45:39.779987 24716 consensus_queue.cc:237] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "85b4f683f6904c35b3c0e91799295bb3" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 39125 } }
W20240502 20:45:39.790027 24649 consensus_peers.cc:485] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77 -> Peer 85b4f683f6904c35b3c0e91799295bb3 (127.22.45.71:39125): Couldn't send request to peer 85b4f683f6904c35b3c0e91799295bb3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3b6e517be2fe4c79bce310d8786554f4. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:45:40.257191 26650 ts_tablet_manager.cc:902] T 3b6e517be2fe4c79bce310d8786554f4 P 85b4f683f6904c35b3c0e91799295bb3: Initiating tablet copy from peer 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65:36905)
I20240502 20:45:40.257910 26650 tablet_copy_client.cc:308] T 3b6e517be2fe4c79bce310d8786554f4 P 85b4f683f6904c35b3c0e91799295bb3: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.65:36905
I20240502 20:45:40.261482 24736 tablet_copy_service.cc:140] P 1c52358e8e2f4ffe992cafe0f1a95a77: Received BeginTabletCopySession request for tablet 3b6e517be2fe4c79bce310d8786554f4 from peer 85b4f683f6904c35b3c0e91799295bb3 ({username='slave'} at 127.22.45.71:42845)
I20240502 20:45:40.261590 24736 tablet_copy_service.cc:161] P 1c52358e8e2f4ffe992cafe0f1a95a77: Beginning new tablet copy session on tablet 3b6e517be2fe4c79bce310d8786554f4 from peer 85b4f683f6904c35b3c0e91799295bb3 at {username='slave'} at 127.22.45.71:42845: session id = 85b4f683f6904c35b3c0e91799295bb3-3b6e517be2fe4c79bce310d8786554f4
I20240502 20:45:40.262497 24736 tablet_copy_source_session.cc:205] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:45:40.265532 26650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b6e517be2fe4c79bce310d8786554f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:40.270835 26650 tablet_copy_client.cc:783] T 3b6e517be2fe4c79bce310d8786554f4 P 85b4f683f6904c35b3c0e91799295bb3: tablet copy: Starting download of 0 data blocks...
I20240502 20:45:40.271008 26650 tablet_copy_client.cc:647] T 3b6e517be2fe4c79bce310d8786554f4 P 85b4f683f6904c35b3c0e91799295bb3: tablet copy: Starting download of 1 WAL segments...
I20240502 20:45:40.272300 26650 tablet_copy_client.cc:523] T 3b6e517be2fe4c79bce310d8786554f4 P 85b4f683f6904c35b3c0e91799295bb3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:45:40.273679 26650 tablet_bootstrap.cc:492] T 3b6e517be2fe4c79bce310d8786554f4 P 85b4f683f6904c35b3c0e91799295bb3: Bootstrap starting.
I20240502 20:45:40.278944 26650 log.cc:824] T 3b6e517be2fe4c79bce310d8786554f4 P 85b4f683f6904c35b3c0e91799295bb3: Log is configured to *not* fsync() on all Append() calls
I20240502 20:45:40.296967 26650 tablet_bootstrap.cc:492] T 3b6e517be2fe4c79bce310d8786554f4 P 85b4f683f6904c35b3c0e91799295bb3: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 20:45:40.297323 26650 tablet_bootstrap.cc:492] T 3b6e517be2fe4c79bce310d8786554f4 P 85b4f683f6904c35b3c0e91799295bb3: Bootstrap complete.
I20240502 20:45:40.297591 26650 ts_tablet_manager.cc:1367] T 3b6e517be2fe4c79bce310d8786554f4 P 85b4f683f6904c35b3c0e91799295bb3: Time spent bootstrapping tablet: real 0.024s user 0.022s sys 0.000s
I20240502 20:45:40.299346 26650 raft_consensus.cc:348] T 3b6e517be2fe4c79bce310d8786554f4 P 85b4f683f6904c35b3c0e91799295bb3 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:40.299834 26650 raft_consensus.cc:729] T 3b6e517be2fe4c79bce310d8786554f4 P 85b4f683f6904c35b3c0e91799295bb3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85b4f683f6904c35b3c0e91799295bb3, State: Initialized, Role: FOLLOWER
I20240502 20:45:40.300096 26650 consensus_queue.cc:260] T 3b6e517be2fe4c79bce310d8786554f4 P 85b4f683f6904c35b3c0e91799295bb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "85b4f683f6904c35b3c0e91799295bb3" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 39125 } }
I20240502 20:45:40.300750 26650 ts_tablet_manager.cc:1398] T 3b6e517be2fe4c79bce310d8786554f4 P 85b4f683f6904c35b3c0e91799295bb3: Time spent starting tablet: real 0.003s user 0.006s sys 0.000s
I20240502 20:45:40.301306 24736 tablet_copy_service.cc:342] P 1c52358e8e2f4ffe992cafe0f1a95a77: Request end of tablet copy session 85b4f683f6904c35b3c0e91799295bb3-3b6e517be2fe4c79bce310d8786554f4 received from {username='slave'} at 127.22.45.71:42845
I20240502 20:45:40.301380 24736 tablet_copy_service.cc:433] P 1c52358e8e2f4ffe992cafe0f1a95a77: ending tablet copy session 85b4f683f6904c35b3c0e91799295bb3-3b6e517be2fe4c79bce310d8786554f4 on tablet 3b6e517be2fe4c79bce310d8786554f4 with peer 85b4f683f6904c35b3c0e91799295bb3
I20240502 20:45:40.808792 26389 raft_consensus.cc:1203] T 3b6e517be2fe4c79bce310d8786554f4 P 85b4f683f6904c35b3c0e91799295bb3 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:45:40.809952 26661 raft_consensus.cc:2932] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 85b4f683f6904c35b3c0e91799295bb3 (127.22.45.71) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "85b4f683f6904c35b3c0e91799295bb3" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 39125 } } }
I20240502 20:45:40.810415 26389 raft_consensus.cc:2932] T 3b6e517be2fe4c79bce310d8786554f4 P 85b4f683f6904c35b3c0e91799295bb3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 85b4f683f6904c35b3c0e91799295bb3 (127.22.45.71) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "85b4f683f6904c35b3c0e91799295bb3" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 39125 } } }
I20240502 20:45:40.811321 26661 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:58032 (request call id 0) took 1031 ms. Trace:
I20240502 20:45:40.811436 26661 rpcz_store.cc:276] 0502 20:45:39.779639 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 20:45:39.779678 (+ 39us) service_pool.cc:225] Handling call
0502 20:45:39.780445 (+ 767us) log.cc:842] Serialized 257 byte log entry
0502 20:45:40.811302 (+1030857us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":45,"raft.queue_time_us":620,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":221,"thread_start_us":58,"threads_started":1,"wal-append.queue_time_us":1042,"wal-append.run_cpu_time_us":297,"wal-append.run_wall_time_us":1000358}
I20240502 20:45:40.812875 24564 catalog_manager.cc:5484] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77 reported cstate change: config changed from index -1 to 4, VOTER 85b4f683f6904c35b3c0e91799295bb3 (127.22.45.71) added. New cstate: current_term: 1 leader_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "85b4f683f6904c35b3c0e91799295bb3" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 39125 } } }
I20240502 20:45:40.831696 24716 consensus_queue.cc:237] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "a984b4b59cb6429290bc907a860d50f8" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44809 } }
W20240502 20:45:40.835963 24652 consensus_peers.cc:485] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77 -> Peer a984b4b59cb6429290bc907a860d50f8 (127.22.45.67:44809): Couldn't send request to peer a984b4b59cb6429290bc907a860d50f8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d661eaefcd884d1b947002588ec335ac. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:45:41.209609 26688 ts_tablet_manager.cc:902] T d661eaefcd884d1b947002588ec335ac P a984b4b59cb6429290bc907a860d50f8: Initiating tablet copy from peer 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65:36905)
I20240502 20:45:41.210269 26688 tablet_copy_client.cc:308] T d661eaefcd884d1b947002588ec335ac P a984b4b59cb6429290bc907a860d50f8: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.65:36905
I20240502 20:45:41.213822 24736 tablet_copy_service.cc:140] P 1c52358e8e2f4ffe992cafe0f1a95a77: Received BeginTabletCopySession request for tablet d661eaefcd884d1b947002588ec335ac from peer a984b4b59cb6429290bc907a860d50f8 ({username='slave'} at 127.22.45.67:44261)
I20240502 20:45:41.213917 24736 tablet_copy_service.cc:161] P 1c52358e8e2f4ffe992cafe0f1a95a77: Beginning new tablet copy session on tablet d661eaefcd884d1b947002588ec335ac from peer a984b4b59cb6429290bc907a860d50f8 at {username='slave'} at 127.22.45.67:44261: session id = a984b4b59cb6429290bc907a860d50f8-d661eaefcd884d1b947002588ec335ac
I20240502 20:45:41.214615 24736 tablet_copy_source_session.cc:205] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:45:41.216097 26688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d661eaefcd884d1b947002588ec335ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:41.218730 26688 tablet_copy_client.cc:783] T d661eaefcd884d1b947002588ec335ac P a984b4b59cb6429290bc907a860d50f8: tablet copy: Starting download of 0 data blocks...
I20240502 20:45:41.218907 26688 tablet_copy_client.cc:647] T d661eaefcd884d1b947002588ec335ac P a984b4b59cb6429290bc907a860d50f8: tablet copy: Starting download of 1 WAL segments...
I20240502 20:45:41.220137 26688 tablet_copy_client.cc:523] T d661eaefcd884d1b947002588ec335ac P a984b4b59cb6429290bc907a860d50f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:45:41.221212 26688 tablet_bootstrap.cc:492] T d661eaefcd884d1b947002588ec335ac P a984b4b59cb6429290bc907a860d50f8: Bootstrap starting.
I20240502 20:45:41.240715 26688 tablet_bootstrap.cc:492] T d661eaefcd884d1b947002588ec335ac P a984b4b59cb6429290bc907a860d50f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 20:45:41.240990 26688 tablet_bootstrap.cc:492] T d661eaefcd884d1b947002588ec335ac P a984b4b59cb6429290bc907a860d50f8: Bootstrap complete.
I20240502 20:45:41.241132 26688 ts_tablet_manager.cc:1367] T d661eaefcd884d1b947002588ec335ac P a984b4b59cb6429290bc907a860d50f8: Time spent bootstrapping tablet: real 0.020s user 0.018s sys 0.000s
I20240502 20:45:41.241576 26688 raft_consensus.cc:348] T d661eaefcd884d1b947002588ec335ac P a984b4b59cb6429290bc907a860d50f8 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:41.241822 26688 raft_consensus.cc:729] T d661eaefcd884d1b947002588ec335ac P a984b4b59cb6429290bc907a860d50f8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a984b4b59cb6429290bc907a860d50f8, State: Initialized, Role: FOLLOWER
I20240502 20:45:41.241930 26688 consensus_queue.cc:260] T d661eaefcd884d1b947002588ec335ac P a984b4b59cb6429290bc907a860d50f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "a984b4b59cb6429290bc907a860d50f8" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44809 } }
I20240502 20:45:41.242265 26688 ts_tablet_manager.cc:1398] T d661eaefcd884d1b947002588ec335ac P a984b4b59cb6429290bc907a860d50f8: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 20:45:41.243100 24736 tablet_copy_service.cc:342] P 1c52358e8e2f4ffe992cafe0f1a95a77: Request end of tablet copy session a984b4b59cb6429290bc907a860d50f8-d661eaefcd884d1b947002588ec335ac received from {username='slave'} at 127.22.45.67:44261
I20240502 20:45:41.243186 24736 tablet_copy_service.cc:433] P 1c52358e8e2f4ffe992cafe0f1a95a77: ending tablet copy session a984b4b59cb6429290bc907a860d50f8-d661eaefcd884d1b947002588ec335ac on tablet d661eaefcd884d1b947002588ec335ac with peer a984b4b59cb6429290bc907a860d50f8
I20240502 20:45:41.684798 25854 raft_consensus.cc:1203] T d661eaefcd884d1b947002588ec335ac P a984b4b59cb6429290bc907a860d50f8 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:45:41.685892 26696 raft_consensus.cc:2932] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a984b4b59cb6429290bc907a860d50f8 (127.22.45.67) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "a984b4b59cb6429290bc907a860d50f8" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44809 } } }
I20240502 20:45:41.686384 25854 raft_consensus.cc:2932] T d661eaefcd884d1b947002588ec335ac P a984b4b59cb6429290bc907a860d50f8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a984b4b59cb6429290bc907a860d50f8 (127.22.45.67) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "a984b4b59cb6429290bc907a860d50f8" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44809 } } }
I20240502 20:45:41.688995 24572 catalog_manager.cc:5484] T d661eaefcd884d1b947002588ec335ac P a984b4b59cb6429290bc907a860d50f8 reported cstate change: config changed from index -1 to 4, VOTER a984b4b59cb6429290bc907a860d50f8 (127.22.45.67) added. New cstate: current_term: 1 leader_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "a984b4b59cb6429290bc907a860d50f8" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44809 } } }
I20240502 20:45:41.704279 24848 consensus_queue.cc:237] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "f6dd93d055454bc5965b407745c020ff" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 36617 } }
W20240502 20:45:41.705531 24783 consensus_peers.cc:485] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4 -> Peer f6dd93d055454bc5965b407745c020ff (127.22.45.70:36617): Couldn't send request to peer f6dd93d055454bc5965b407745c020ff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 61dc146b84774046980dc8fdbf8e632f. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:45:42.266487 26719 ts_tablet_manager.cc:902] T 61dc146b84774046980dc8fdbf8e632f P f6dd93d055454bc5965b407745c020ff: Initiating tablet copy from peer f926d9d9d2314c2b90437934356f66d4 (127.22.45.66:46041)
I20240502 20:45:42.266930 26719 tablet_copy_client.cc:308] T 61dc146b84774046980dc8fdbf8e632f P f6dd93d055454bc5965b407745c020ff: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.66:46041
I20240502 20:45:42.267414 24868 tablet_copy_service.cc:140] P f926d9d9d2314c2b90437934356f66d4: Received BeginTabletCopySession request for tablet 61dc146b84774046980dc8fdbf8e632f from peer f6dd93d055454bc5965b407745c020ff ({username='slave'} at 127.22.45.70:40711)
I20240502 20:45:42.267516 24868 tablet_copy_service.cc:161] P f926d9d9d2314c2b90437934356f66d4: Beginning new tablet copy session on tablet 61dc146b84774046980dc8fdbf8e632f from peer f6dd93d055454bc5965b407745c020ff at {username='slave'} at 127.22.45.70:40711: session id = f6dd93d055454bc5965b407745c020ff-61dc146b84774046980dc8fdbf8e632f
I20240502 20:45:42.268260 24868 tablet_copy_source_session.cc:205] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:45:42.269068 26719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61dc146b84774046980dc8fdbf8e632f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:42.270803 26719 tablet_copy_client.cc:783] T 61dc146b84774046980dc8fdbf8e632f P f6dd93d055454bc5965b407745c020ff: tablet copy: Starting download of 0 data blocks...
I20240502 20:45:42.270952 26719 tablet_copy_client.cc:647] T 61dc146b84774046980dc8fdbf8e632f P f6dd93d055454bc5965b407745c020ff: tablet copy: Starting download of 1 WAL segments...
I20240502 20:45:42.271961 26719 tablet_copy_client.cc:523] T 61dc146b84774046980dc8fdbf8e632f P f6dd93d055454bc5965b407745c020ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:45:42.272894 26719 tablet_bootstrap.cc:492] T 61dc146b84774046980dc8fdbf8e632f P f6dd93d055454bc5965b407745c020ff: Bootstrap starting.
I20240502 20:45:42.290009 26719 tablet_bootstrap.cc:492] T 61dc146b84774046980dc8fdbf8e632f P f6dd93d055454bc5965b407745c020ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 20:45:42.290246 26719 tablet_bootstrap.cc:492] T 61dc146b84774046980dc8fdbf8e632f P f6dd93d055454bc5965b407745c020ff: Bootstrap complete.
I20240502 20:45:42.290396 26719 ts_tablet_manager.cc:1367] T 61dc146b84774046980dc8fdbf8e632f P f6dd93d055454bc5965b407745c020ff: Time spent bootstrapping tablet: real 0.018s user 0.015s sys 0.002s
I20240502 20:45:42.290899 26719 raft_consensus.cc:348] T 61dc146b84774046980dc8fdbf8e632f P f6dd93d055454bc5965b407745c020ff [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:42.291086 26719 raft_consensus.cc:729] T 61dc146b84774046980dc8fdbf8e632f P f6dd93d055454bc5965b407745c020ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6dd93d055454bc5965b407745c020ff, State: Initialized, Role: FOLLOWER
I20240502 20:45:42.291177 26719 consensus_queue.cc:260] T 61dc146b84774046980dc8fdbf8e632f P f6dd93d055454bc5965b407745c020ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "f6dd93d055454bc5965b407745c020ff" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 36617 } }
I20240502 20:45:42.291500 26719 ts_tablet_manager.cc:1398] T 61dc146b84774046980dc8fdbf8e632f P f6dd93d055454bc5965b407745c020ff: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 20:45:42.291983 24868 tablet_copy_service.cc:342] P f926d9d9d2314c2b90437934356f66d4: Request end of tablet copy session f6dd93d055454bc5965b407745c020ff-61dc146b84774046980dc8fdbf8e632f received from {username='slave'} at 127.22.45.70:40711
I20240502 20:45:42.292057 24868 tablet_copy_service.cc:433] P f926d9d9d2314c2b90437934356f66d4: ending tablet copy session f6dd93d055454bc5965b407745c020ff-61dc146b84774046980dc8fdbf8e632f on tablet 61dc146b84774046980dc8fdbf8e632f with peer f6dd93d055454bc5965b407745c020ff
I20240502 20:45:42.747715 26257 raft_consensus.cc:1203] T 61dc146b84774046980dc8fdbf8e632f P f6dd93d055454bc5965b407745c020ff [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:45:42.748584 26726 raft_consensus.cc:2932] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f6dd93d055454bc5965b407745c020ff (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "f6dd93d055454bc5965b407745c020ff" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 36617 } } }
I20240502 20:45:42.749028 26257 raft_consensus.cc:2932] T 61dc146b84774046980dc8fdbf8e632f P f6dd93d055454bc5965b407745c020ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f6dd93d055454bc5965b407745c020ff (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "f6dd93d055454bc5965b407745c020ff" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 36617 } } }
I20240502 20:45:42.750102 26726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:44240 (request call id 0) took 1046 ms. Trace:
I20240502 20:45:42.750197 26726 rpcz_store.cc:276] 0502 20:45:41.703991 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 20:45:41.704039 (+ 48us) service_pool.cc:225] Handling call
0502 20:45:41.704634 (+ 595us) log.cc:842] Serialized 257 byte log entry
0502 20:45:42.750083 (+1045449us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":30,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":138,"spinlock_wait_cycles":10624,"thread_start_us":61,"threads_started":1,"wal-append.queue_time_us":127,"wal-append.run_cpu_time_us":286,"wal-append.run_wall_time_us":1000351}
I20240502 20:45:42.751214 24564 catalog_manager.cc:5484] T 61dc146b84774046980dc8fdbf8e632f P f6dd93d055454bc5965b407745c020ff reported cstate change: config changed from index -1 to 4, VOTER f6dd93d055454bc5965b407745c020ff (127.22.45.70) added. New cstate: current_term: 1 leader_uuid: "f926d9d9d2314c2b90437934356f66d4" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "f6dd93d055454bc5965b407745c020ff" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 36617 } } }
I20240502 20:45:42.767302 24716 consensus_queue.cc:237] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "eef5daac82024c449fc8b0dc82bd428c" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 37677 } }
W20240502 20:45:42.768517 24651 consensus_peers.cc:485] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77 -> Peer eef5daac82024c449fc8b0dc82bd428c (127.22.45.69:37677): Couldn't send request to peer eef5daac82024c449fc8b0dc82bd428c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7ed23be7cabc44b4ad4db0ee4f2e5392. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:45:43.353353 26751 ts_tablet_manager.cc:902] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P eef5daac82024c449fc8b0dc82bd428c: Initiating tablet copy from peer 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65:36905)
I20240502 20:45:43.353847 26751 tablet_copy_client.cc:308] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P eef5daac82024c449fc8b0dc82bd428c: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.65:36905
I20240502 20:45:43.354336 24736 tablet_copy_service.cc:140] P 1c52358e8e2f4ffe992cafe0f1a95a77: Received BeginTabletCopySession request for tablet 7ed23be7cabc44b4ad4db0ee4f2e5392 from peer eef5daac82024c449fc8b0dc82bd428c ({username='slave'} at 127.22.45.69:36663)
I20240502 20:45:43.354450 24736 tablet_copy_service.cc:161] P 1c52358e8e2f4ffe992cafe0f1a95a77: Beginning new tablet copy session on tablet 7ed23be7cabc44b4ad4db0ee4f2e5392 from peer eef5daac82024c449fc8b0dc82bd428c at {username='slave'} at 127.22.45.69:36663: session id = eef5daac82024c449fc8b0dc82bd428c-7ed23be7cabc44b4ad4db0ee4f2e5392
I20240502 20:45:43.355207 24736 tablet_copy_source_session.cc:205] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:45:43.356043 26751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ed23be7cabc44b4ad4db0ee4f2e5392. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:43.357829 26751 tablet_copy_client.cc:783] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P eef5daac82024c449fc8b0dc82bd428c: tablet copy: Starting download of 0 data blocks...
I20240502 20:45:43.357988 26751 tablet_copy_client.cc:647] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P eef5daac82024c449fc8b0dc82bd428c: tablet copy: Starting download of 1 WAL segments...
I20240502 20:45:43.359055 26751 tablet_copy_client.cc:523] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P eef5daac82024c449fc8b0dc82bd428c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:45:43.360008 26751 tablet_bootstrap.cc:492] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P eef5daac82024c449fc8b0dc82bd428c: Bootstrap starting.
I20240502 20:45:43.376549 26751 tablet_bootstrap.cc:492] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P eef5daac82024c449fc8b0dc82bd428c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 20:45:43.376781 26751 tablet_bootstrap.cc:492] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P eef5daac82024c449fc8b0dc82bd428c: Bootstrap complete.
I20240502 20:45:43.376920 26751 ts_tablet_manager.cc:1367] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P eef5daac82024c449fc8b0dc82bd428c: Time spent bootstrapping tablet: real 0.017s user 0.011s sys 0.005s
I20240502 20:45:43.377363 26751 raft_consensus.cc:348] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P eef5daac82024c449fc8b0dc82bd428c [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:43.377559 26751 raft_consensus.cc:729] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P eef5daac82024c449fc8b0dc82bd428c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eef5daac82024c449fc8b0dc82bd428c, State: Initialized, Role: FOLLOWER
I20240502 20:45:43.377650 26751 consensus_queue.cc:260] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P eef5daac82024c449fc8b0dc82bd428c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "eef5daac82024c449fc8b0dc82bd428c" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 37677 } }
I20240502 20:45:43.377969 26751 ts_tablet_manager.cc:1398] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P eef5daac82024c449fc8b0dc82bd428c: Time spent starting tablet: real 0.001s user 0.003s sys 0.001s
I20240502 20:45:43.378556 24736 tablet_copy_service.cc:342] P 1c52358e8e2f4ffe992cafe0f1a95a77: Request end of tablet copy session eef5daac82024c449fc8b0dc82bd428c-7ed23be7cabc44b4ad4db0ee4f2e5392 received from {username='slave'} at 127.22.45.69:36663
I20240502 20:45:43.378654 24736 tablet_copy_service.cc:433] P 1c52358e8e2f4ffe992cafe0f1a95a77: ending tablet copy session eef5daac82024c449fc8b0dc82bd428c-7ed23be7cabc44b4ad4db0ee4f2e5392 on tablet 7ed23be7cabc44b4ad4db0ee4f2e5392 with peer eef5daac82024c449fc8b0dc82bd428c
I20240502 20:45:43.821480 26125 raft_consensus.cc:1203] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P eef5daac82024c449fc8b0dc82bd428c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:45:43.822340 26758 raft_consensus.cc:2932] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER eef5daac82024c449fc8b0dc82bd428c (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "eef5daac82024c449fc8b0dc82bd428c" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 37677 } } }
I20240502 20:45:43.822881 26125 raft_consensus.cc:2932] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P eef5daac82024c449fc8b0dc82bd428c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER eef5daac82024c449fc8b0dc82bd428c (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "eef5daac82024c449fc8b0dc82bd428c" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 37677 } } }
I20240502 20:45:43.823941 26758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:58080 (request call id 0) took 1056 ms. Trace:
I20240502 20:45:43.824000 26758 rpcz_store.cc:276] 0502 20:45:42.767014 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 20:45:42.767062 (+ 48us) service_pool.cc:225] Handling call
0502 20:45:42.767632 (+ 570us) log.cc:842] Serialized 257 byte log entry
0502 20:45:43.823915 (+1056283us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":42,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"thread_start_us":62,"threads_started":1,"wal-append.queue_time_us":118,"wal-append.run_cpu_time_us":275,"wal-append.run_wall_time_us":1000336}
I20240502 20:45:43.825237 24564 catalog_manager.cc:5484] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P eef5daac82024c449fc8b0dc82bd428c reported cstate change: config changed from index -1 to 4, VOTER eef5daac82024c449fc8b0dc82bd428c (127.22.45.69) added. New cstate: current_term: 1 leader_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "eef5daac82024c449fc8b0dc82bd428c" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 37677 } } }
I20240502 20:45:43.841006 24716 consensus_queue.cc:237] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "bb697f05a8394524a411ece19fe608dc" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 39501 } }
W20240502 20:45:43.844856 24650 consensus_peers.cc:485] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77 -> Peer bb697f05a8394524a411ece19fe608dc (127.22.45.68:39501): Couldn't send request to peer bb697f05a8394524a411ece19fe608dc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77f37012797e41cb9aafd5ab00d049ab. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:45:44.248776 26785 ts_tablet_manager.cc:902] T 77f37012797e41cb9aafd5ab00d049ab P bb697f05a8394524a411ece19fe608dc: Initiating tablet copy from peer 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65:36905)
I20240502 20:45:44.249318 26785 tablet_copy_client.cc:308] T 77f37012797e41cb9aafd5ab00d049ab P bb697f05a8394524a411ece19fe608dc: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.65:36905
I20240502 20:45:44.253065 24736 tablet_copy_service.cc:140] P 1c52358e8e2f4ffe992cafe0f1a95a77: Received BeginTabletCopySession request for tablet 77f37012797e41cb9aafd5ab00d049ab from peer bb697f05a8394524a411ece19fe608dc ({username='slave'} at 127.22.45.68:51367)
I20240502 20:45:44.253157 24736 tablet_copy_service.cc:161] P 1c52358e8e2f4ffe992cafe0f1a95a77: Beginning new tablet copy session on tablet 77f37012797e41cb9aafd5ab00d049ab from peer bb697f05a8394524a411ece19fe608dc at {username='slave'} at 127.22.45.68:51367: session id = bb697f05a8394524a411ece19fe608dc-77f37012797e41cb9aafd5ab00d049ab
I20240502 20:45:44.253840 24736 tablet_copy_source_session.cc:205] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:45:44.254639 26785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77f37012797e41cb9aafd5ab00d049ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:44.256301 26785 tablet_copy_client.cc:783] T 77f37012797e41cb9aafd5ab00d049ab P bb697f05a8394524a411ece19fe608dc: tablet copy: Starting download of 0 data blocks...
I20240502 20:45:44.256450 26785 tablet_copy_client.cc:647] T 77f37012797e41cb9aafd5ab00d049ab P bb697f05a8394524a411ece19fe608dc: tablet copy: Starting download of 1 WAL segments...
I20240502 20:45:44.257426 26785 tablet_copy_client.cc:523] T 77f37012797e41cb9aafd5ab00d049ab P bb697f05a8394524a411ece19fe608dc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:45:44.258325 26785 tablet_bootstrap.cc:492] T 77f37012797e41cb9aafd5ab00d049ab P bb697f05a8394524a411ece19fe608dc: Bootstrap starting.
I20240502 20:45:44.275506 26785 tablet_bootstrap.cc:492] T 77f37012797e41cb9aafd5ab00d049ab P bb697f05a8394524a411ece19fe608dc: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 20:45:44.275727 26785 tablet_bootstrap.cc:492] T 77f37012797e41cb9aafd5ab00d049ab P bb697f05a8394524a411ece19fe608dc: Bootstrap complete.
I20240502 20:45:44.275882 26785 ts_tablet_manager.cc:1367] T 77f37012797e41cb9aafd5ab00d049ab P bb697f05a8394524a411ece19fe608dc: Time spent bootstrapping tablet: real 0.018s user 0.014s sys 0.003s
I20240502 20:45:44.276304 26785 raft_consensus.cc:348] T 77f37012797e41cb9aafd5ab00d049ab P bb697f05a8394524a411ece19fe608dc [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:44.276500 26785 raft_consensus.cc:729] T 77f37012797e41cb9aafd5ab00d049ab P bb697f05a8394524a411ece19fe608dc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb697f05a8394524a411ece19fe608dc, State: Initialized, Role: FOLLOWER
I20240502 20:45:44.276587 26785 consensus_queue.cc:260] T 77f37012797e41cb9aafd5ab00d049ab P bb697f05a8394524a411ece19fe608dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "bb697f05a8394524a411ece19fe608dc" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 39501 } }
I20240502 20:45:44.276875 26785 ts_tablet_manager.cc:1398] T 77f37012797e41cb9aafd5ab00d049ab P bb697f05a8394524a411ece19fe608dc: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 20:45:44.277359 24736 tablet_copy_service.cc:342] P 1c52358e8e2f4ffe992cafe0f1a95a77: Request end of tablet copy session bb697f05a8394524a411ece19fe608dc-77f37012797e41cb9aafd5ab00d049ab received from {username='slave'} at 127.22.45.68:51367
I20240502 20:45:44.277446 24736 tablet_copy_service.cc:433] P 1c52358e8e2f4ffe992cafe0f1a95a77: ending tablet copy session bb697f05a8394524a411ece19fe608dc-77f37012797e41cb9aafd5ab00d049ab on tablet 77f37012797e41cb9aafd5ab00d049ab with peer bb697f05a8394524a411ece19fe608dc
I20240502 20:45:44.686576 25993 raft_consensus.cc:1203] T 77f37012797e41cb9aafd5ab00d049ab P bb697f05a8394524a411ece19fe608dc [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:45:44.687412 26793 raft_consensus.cc:2932] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER bb697f05a8394524a411ece19fe608dc (127.22.45.68) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "bb697f05a8394524a411ece19fe608dc" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 39501 } } }
I20240502 20:45:44.687896 25993 raft_consensus.cc:2932] T 77f37012797e41cb9aafd5ab00d049ab P bb697f05a8394524a411ece19fe608dc [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER bb697f05a8394524a411ece19fe608dc (127.22.45.68) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "bb697f05a8394524a411ece19fe608dc" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 39501 } } }
I20240502 20:45:44.690184 24572 catalog_manager.cc:5484] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77 reported cstate change: config changed from index -1 to 4, VOTER bb697f05a8394524a411ece19fe608dc (127.22.45.68) added. New cstate: current_term: 1 leader_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "bb697f05a8394524a411ece19fe608dc" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 39501 } } }
I20240502 20:45:44.706859 24848 consensus_queue.cc:237] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "85b4f683f6904c35b3c0e91799295bb3" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 39125 } }
W20240502 20:45:44.710695 24781 consensus_peers.cc:485] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4 -> Peer 85b4f683f6904c35b3c0e91799295bb3 (127.22.45.71:39125): Couldn't send request to peer 85b4f683f6904c35b3c0e91799295bb3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f258bd1f1bd14f99bed69e112cb09358. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:45:45.230679 26818 ts_tablet_manager.cc:902] T f258bd1f1bd14f99bed69e112cb09358 P 85b4f683f6904c35b3c0e91799295bb3: Initiating tablet copy from peer f926d9d9d2314c2b90437934356f66d4 (127.22.45.66:46041)
I20240502 20:45:45.231143 26818 tablet_copy_client.cc:308] T f258bd1f1bd14f99bed69e112cb09358 P 85b4f683f6904c35b3c0e91799295bb3: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.66:46041
I20240502 20:45:45.234524 24868 tablet_copy_service.cc:140] P f926d9d9d2314c2b90437934356f66d4: Received BeginTabletCopySession request for tablet f258bd1f1bd14f99bed69e112cb09358 from peer 85b4f683f6904c35b3c0e91799295bb3 ({username='slave'} at 127.22.45.71:48679)
I20240502 20:45:45.234656 24868 tablet_copy_service.cc:161] P f926d9d9d2314c2b90437934356f66d4: Beginning new tablet copy session on tablet f258bd1f1bd14f99bed69e112cb09358 from peer 85b4f683f6904c35b3c0e91799295bb3 at {username='slave'} at 127.22.45.71:48679: session id = 85b4f683f6904c35b3c0e91799295bb3-f258bd1f1bd14f99bed69e112cb09358
I20240502 20:45:45.235455 24868 tablet_copy_source_session.cc:205] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:45:45.236249 26818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f258bd1f1bd14f99bed69e112cb09358. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:45.237856 26818 tablet_copy_client.cc:783] T f258bd1f1bd14f99bed69e112cb09358 P 85b4f683f6904c35b3c0e91799295bb3: tablet copy: Starting download of 0 data blocks...
I20240502 20:45:45.238016 26818 tablet_copy_client.cc:647] T f258bd1f1bd14f99bed69e112cb09358 P 85b4f683f6904c35b3c0e91799295bb3: tablet copy: Starting download of 1 WAL segments...
I20240502 20:45:45.238982 26818 tablet_copy_client.cc:523] T f258bd1f1bd14f99bed69e112cb09358 P 85b4f683f6904c35b3c0e91799295bb3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:45:45.239872 26818 tablet_bootstrap.cc:492] T f258bd1f1bd14f99bed69e112cb09358 P 85b4f683f6904c35b3c0e91799295bb3: Bootstrap starting.
I20240502 20:45:45.255787 26818 tablet_bootstrap.cc:492] T f258bd1f1bd14f99bed69e112cb09358 P 85b4f683f6904c35b3c0e91799295bb3: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 20:45:45.256006 26818 tablet_bootstrap.cc:492] T f258bd1f1bd14f99bed69e112cb09358 P 85b4f683f6904c35b3c0e91799295bb3: Bootstrap complete.
I20240502 20:45:45.256126 26818 ts_tablet_manager.cc:1367] T f258bd1f1bd14f99bed69e112cb09358 P 85b4f683f6904c35b3c0e91799295bb3: Time spent bootstrapping tablet: real 0.016s user 0.011s sys 0.005s
I20240502 20:45:45.256546 26818 raft_consensus.cc:348] T f258bd1f1bd14f99bed69e112cb09358 P 85b4f683f6904c35b3c0e91799295bb3 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:45.256723 26818 raft_consensus.cc:729] T f258bd1f1bd14f99bed69e112cb09358 P 85b4f683f6904c35b3c0e91799295bb3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85b4f683f6904c35b3c0e91799295bb3, State: Initialized, Role: FOLLOWER
I20240502 20:45:45.256820 26818 consensus_queue.cc:260] T f258bd1f1bd14f99bed69e112cb09358 P 85b4f683f6904c35b3c0e91799295bb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "85b4f683f6904c35b3c0e91799295bb3" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 39125 } }
I20240502 20:45:45.257113 26818 ts_tablet_manager.cc:1398] T f258bd1f1bd14f99bed69e112cb09358 P 85b4f683f6904c35b3c0e91799295bb3: Time spent starting tablet: real 0.001s user 0.002s sys 0.001s
I20240502 20:45:45.257586 24868 tablet_copy_service.cc:342] P f926d9d9d2314c2b90437934356f66d4: Request end of tablet copy session 85b4f683f6904c35b3c0e91799295bb3-f258bd1f1bd14f99bed69e112cb09358 received from {username='slave'} at 127.22.45.71:48679
I20240502 20:45:45.257653 24868 tablet_copy_service.cc:433] P f926d9d9d2314c2b90437934356f66d4: ending tablet copy session 85b4f683f6904c35b3c0e91799295bb3-f258bd1f1bd14f99bed69e112cb09358 on tablet f258bd1f1bd14f99bed69e112cb09358 with peer 85b4f683f6904c35b3c0e91799295bb3
I20240502 20:45:45.607409 26389 raft_consensus.cc:1203] T f258bd1f1bd14f99bed69e112cb09358 P 85b4f683f6904c35b3c0e91799295bb3 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:45:45.608238 26827 raft_consensus.cc:2932] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 85b4f683f6904c35b3c0e91799295bb3 (127.22.45.71) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "85b4f683f6904c35b3c0e91799295bb3" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 39125 } } }
I20240502 20:45:45.608692 26389 raft_consensus.cc:2932] T f258bd1f1bd14f99bed69e112cb09358 P 85b4f683f6904c35b3c0e91799295bb3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 85b4f683f6904c35b3c0e91799295bb3 (127.22.45.71) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "85b4f683f6904c35b3c0e91799295bb3" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 39125 } } }
I20240502 20:45:45.611181 24564 catalog_manager.cc:5484] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4 reported cstate change: config changed from index -1 to 4, VOTER 85b4f683f6904c35b3c0e91799295bb3 (127.22.45.71) added. New cstate: current_term: 1 leader_uuid: "f926d9d9d2314c2b90437934356f66d4" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "85b4f683f6904c35b3c0e91799295bb3" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 39125 } } }
I20240502 20:45:45.665807 26369 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:45.669175 25973 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:45.670620 24696 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:45:45.674571 25834 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:45.674714 26105 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:45.677856 24828 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:45:45.695968 26237 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:45.717250 24848 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f258bd1f1bd14f99bed69e112cb09358"
dest_uuid: "f926d9d9d2314c2b90437934356f66d4"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:44284
I20240502 20:45:45.717427 24848 raft_consensus.cc:595] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Received request to transfer leadership
I20240502 20:45:45.764003 26369 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:45.764287 24696 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:45:45.770692 24828 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:45:45.775240 26105 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:45.775514 25973 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:45.776571 25834 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:45.777981 26237 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:45.871820 26369 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:45.871978 24696 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:45:45.872257 26105 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:45.873855 26237 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:45.874281 25973 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:45.891456 24828 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:45:45.900983 25834 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:45.921727 24716 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "77f37012797e41cb9aafd5ab00d049ab"
dest_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:58156
I20240502 20:45:45.921888 24716 raft_consensus.cc:595] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Received request to transfer leadership
I20240502 20:45:45.970156 25973 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:45.971962 25834 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:45.973898 24696 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:45:45.979436 26105 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:45.981272 26369 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:45.982803 26237 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:45.986760 24828 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:45:46.070881 26369 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:46.074513 24696 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:45:46.075850 26237 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:46.082873 25834 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.085707 25973 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:46.095791 26105 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:46.097163 26726 raft_consensus.cc:979] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4: : Instructing follower 85b4f683f6904c35b3c0e91799295bb3 to start an election
I20240502 20:45:46.097239 26726 raft_consensus.cc:1067] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Signalling peer 85b4f683f6904c35b3c0e91799295bb3 to start an election
I20240502 20:45:46.097888 26389 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f258bd1f1bd14f99bed69e112cb09358"
dest_uuid: "85b4f683f6904c35b3c0e91799295bb3"
from {username='slave'} at 127.22.45.66:48089
I20240502 20:45:46.098057 26389 raft_consensus.cc:482] T f258bd1f1bd14f99bed69e112cb09358 P 85b4f683f6904c35b3c0e91799295bb3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:45:46.098145 26389 raft_consensus.cc:3037] T f258bd1f1bd14f99bed69e112cb09358 P 85b4f683f6904c35b3c0e91799295bb3 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:45:46.099152 26389 raft_consensus.cc:504] T f258bd1f1bd14f99bed69e112cb09358 P 85b4f683f6904c35b3c0e91799295bb3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "85b4f683f6904c35b3c0e91799295bb3" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 39125 } }
I20240502 20:45:46.100104 26389 leader_election.cc:288] T f258bd1f1bd14f99bed69e112cb09358 P 85b4f683f6904c35b3c0e91799295bb3 [CANDIDATE]: Term 2 election: Requested vote from peers f926d9d9d2314c2b90437934356f66d4 (127.22.45.66:46041)
I20240502 20:45:46.101675 24848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f258bd1f1bd14f99bed69e112cb09358" candidate_uuid: "85b4f683f6904c35b3c0e91799295bb3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f926d9d9d2314c2b90437934356f66d4"
I20240502 20:45:46.101826 24848 raft_consensus.cc:3032] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:45:46.101861 24848 raft_consensus.cc:729] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f926d9d9d2314c2b90437934356f66d4, State: Running, Role: LEADER
I20240502 20:45:46.101986 24848 consensus_queue.cc:260] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "85b4f683f6904c35b3c0e91799295bb3" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 39125 } }
I20240502 20:45:46.102207 24848 raft_consensus.cc:3037] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:45:46.103132 24848 raft_consensus.cc:2454] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85b4f683f6904c35b3c0e91799295bb3 in term 2.
I20240502 20:45:46.101682 24828 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:45:46.103502 26323 leader_election.cc:302] T f258bd1f1bd14f99bed69e112cb09358 P 85b4f683f6904c35b3c0e91799295bb3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 85b4f683f6904c35b3c0e91799295bb3, f926d9d9d2314c2b90437934356f66d4; no voters:
I20240502 20:45:46.103835 26826 raft_consensus.cc:2781] T f258bd1f1bd14f99bed69e112cb09358 P 85b4f683f6904c35b3c0e91799295bb3 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:45:46.103989 26826 raft_consensus.cc:686] T f258bd1f1bd14f99bed69e112cb09358 P 85b4f683f6904c35b3c0e91799295bb3 [term 2 LEADER]: Becoming Leader. State: Replica: 85b4f683f6904c35b3c0e91799295bb3, State: Running, Role: LEADER
I20240502 20:45:46.104255 26826 consensus_queue.cc:237] T f258bd1f1bd14f99bed69e112cb09358 P 85b4f683f6904c35b3c0e91799295bb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "85b4f683f6904c35b3c0e91799295bb3" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 39125 } }
I20240502 20:45:46.106222 24565 catalog_manager.cc:5484] T f258bd1f1bd14f99bed69e112cb09358 P 85b4f683f6904c35b3c0e91799295bb3 reported cstate change: term changed from 1 to 2, leader changed from f926d9d9d2314c2b90437934356f66d4 (127.22.45.66) to 85b4f683f6904c35b3c0e91799295bb3 (127.22.45.71). New cstate: current_term: 2 leader_uuid: "85b4f683f6904c35b3c0e91799295bb3" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "85b4f683f6904c35b3c0e91799295bb3" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 39125 } } }
I20240502 20:45:46.126538 24716 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7ed23be7cabc44b4ad4db0ee4f2e5392"
dest_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:58188
I20240502 20:45:46.126655 24716 raft_consensus.cc:595] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Received request to transfer leadership
I20240502 20:45:46.172020 24696 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:45:46.173672 26369 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.178752 25834 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.181623 25973 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:46.186735 26105 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:46.186779 24828 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:45:46.187700 26237 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:46.270344 24696 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:45:46.270747 26369 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.280817 26105 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:46.283818 25834 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.283998 25973 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:46.288671 24828 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:45:46.296747 26237 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:46.319918 24848 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "71e60972a8354e23adb03542b9935769"
dest_uuid: "f926d9d9d2314c2b90437934356f66d4"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:44350
I20240502 20:45:46.320052 24848 raft_consensus.cc:595] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Received request to transfer leadership
I20240502 20:45:46.335599 26696 raft_consensus.cc:979] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77: : Instructing follower bb697f05a8394524a411ece19fe608dc to start an election
I20240502 20:45:46.335711 26696 raft_consensus.cc:1067] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Signalling peer bb697f05a8394524a411ece19fe608dc to start an election
I20240502 20:45:46.336268 25993 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "77f37012797e41cb9aafd5ab00d049ab"
dest_uuid: "bb697f05a8394524a411ece19fe608dc"
from {username='slave'} at 127.22.45.65:40589
I20240502 20:45:46.336459 25993 raft_consensus.cc:482] T 77f37012797e41cb9aafd5ab00d049ab P bb697f05a8394524a411ece19fe608dc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:45:46.336560 25993 raft_consensus.cc:3037] T 77f37012797e41cb9aafd5ab00d049ab P bb697f05a8394524a411ece19fe608dc [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:45:46.337580 25993 raft_consensus.cc:504] T 77f37012797e41cb9aafd5ab00d049ab P bb697f05a8394524a411ece19fe608dc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "bb697f05a8394524a411ece19fe608dc" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 39501 } }
I20240502 20:45:46.338591 25993 leader_election.cc:288] T 77f37012797e41cb9aafd5ab00d049ab P bb697f05a8394524a411ece19fe608dc [CANDIDATE]: Term 2 election: Requested vote from peers 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65:36905)
I20240502 20:45:46.338954 24716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77f37012797e41cb9aafd5ab00d049ab" candidate_uuid: "bb697f05a8394524a411ece19fe608dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77"
I20240502 20:45:46.339100 24716 raft_consensus.cc:3032] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:45:46.339147 24716 raft_consensus.cc:729] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1c52358e8e2f4ffe992cafe0f1a95a77, State: Running, Role: LEADER
I20240502 20:45:46.339294 24716 consensus_queue.cc:260] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "bb697f05a8394524a411ece19fe608dc" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 39501 } }
I20240502 20:45:46.339495 24716 raft_consensus.cc:3037] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:45:46.340309 24716 raft_consensus.cc:2454] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb697f05a8394524a411ece19fe608dc in term 2.
I20240502 20:45:46.340641 25929 leader_election.cc:302] T 77f37012797e41cb9aafd5ab00d049ab P bb697f05a8394524a411ece19fe608dc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1c52358e8e2f4ffe992cafe0f1a95a77, bb697f05a8394524a411ece19fe608dc; no voters:
I20240502 20:45:46.340956 27116 raft_consensus.cc:2781] T 77f37012797e41cb9aafd5ab00d049ab P bb697f05a8394524a411ece19fe608dc [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:45:46.341120 27116 raft_consensus.cc:686] T 77f37012797e41cb9aafd5ab00d049ab P bb697f05a8394524a411ece19fe608dc [term 2 LEADER]: Becoming Leader. State: Replica: bb697f05a8394524a411ece19fe608dc, State: Running, Role: LEADER
I20240502 20:45:46.341410 27116 consensus_queue.cc:237] T 77f37012797e41cb9aafd5ab00d049ab P bb697f05a8394524a411ece19fe608dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "bb697f05a8394524a411ece19fe608dc" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 39501 } }
I20240502 20:45:46.343955 24564 catalog_manager.cc:5484] T 77f37012797e41cb9aafd5ab00d049ab P bb697f05a8394524a411ece19fe608dc reported cstate change: term changed from 1 to 2, leader changed from 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65) to bb697f05a8394524a411ece19fe608dc (127.22.45.68). New cstate: current_term: 2 leader_uuid: "bb697f05a8394524a411ece19fe608dc" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "bb697f05a8394524a411ece19fe608dc" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 39501 } } }
I20240502 20:45:46.370570 26105 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:46.370998 25973 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.372503 25834 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.374501 26369 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.378849 24696 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:45:46.381887 26237 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:46.405154 24828 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:45:46.477993 24696 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:45:46.479545 25834 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.483685 26369 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.483716 26105 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:46.485041 25973 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.486765 26237 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:46.493160 27110 raft_consensus.cc:979] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77: : Instructing follower eef5daac82024c449fc8b0dc82bd428c to start an election
I20240502 20:45:46.493258 27110 raft_consensus.cc:1067] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Signalling peer eef5daac82024c449fc8b0dc82bd428c to start an election
I20240502 20:45:46.493731 26125 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7ed23be7cabc44b4ad4db0ee4f2e5392"
dest_uuid: "eef5daac82024c449fc8b0dc82bd428c"
from {username='slave'} at 127.22.45.65:37859
I20240502 20:45:46.493907 26125 raft_consensus.cc:482] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P eef5daac82024c449fc8b0dc82bd428c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:45:46.493995 26125 raft_consensus.cc:3037] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P eef5daac82024c449fc8b0dc82bd428c [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:45:46.495040 26125 raft_consensus.cc:504] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P eef5daac82024c449fc8b0dc82bd428c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "eef5daac82024c449fc8b0dc82bd428c" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 37677 } }
I20240502 20:45:46.495359 24828 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:45:46.496024 26125 leader_election.cc:288] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P eef5daac82024c449fc8b0dc82bd428c [CANDIDATE]: Term 2 election: Requested vote from peers 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65:36905)
I20240502 20:45:46.496361 24716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed23be7cabc44b4ad4db0ee4f2e5392" candidate_uuid: "eef5daac82024c449fc8b0dc82bd428c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77"
I20240502 20:45:46.496474 24716 raft_consensus.cc:3032] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:45:46.496515 24716 raft_consensus.cc:729] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1c52358e8e2f4ffe992cafe0f1a95a77, State: Running, Role: LEADER
I20240502 20:45:46.496644 24716 consensus_queue.cc:260] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "eef5daac82024c449fc8b0dc82bd428c" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 37677 } }
I20240502 20:45:46.496726 24716 raft_consensus.cc:3037] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:45:46.497601 24716 raft_consensus.cc:2454] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eef5daac82024c449fc8b0dc82bd428c in term 2.
I20240502 20:45:46.498337 26061 leader_election.cc:302] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P eef5daac82024c449fc8b0dc82bd428c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1c52358e8e2f4ffe992cafe0f1a95a77, eef5daac82024c449fc8b0dc82bd428c; no voters:
I20240502 20:45:46.506705 27177 raft_consensus.cc:2781] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P eef5daac82024c449fc8b0dc82bd428c [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:45:46.517058 27177 raft_consensus.cc:686] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P eef5daac82024c449fc8b0dc82bd428c [term 2 LEADER]: Becoming Leader. State: Replica: eef5daac82024c449fc8b0dc82bd428c, State: Running, Role: LEADER
I20240502 20:45:46.517469 27177 consensus_queue.cc:237] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P eef5daac82024c449fc8b0dc82bd428c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "eef5daac82024c449fc8b0dc82bd428c" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 37677 } }
I20240502 20:45:46.519968 24564 catalog_manager.cc:5484] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P eef5daac82024c449fc8b0dc82bd428c reported cstate change: term changed from 1 to 2, leader changed from 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65) to eef5daac82024c449fc8b0dc82bd428c (127.22.45.69). New cstate: current_term: 2 leader_uuid: "eef5daac82024c449fc8b0dc82bd428c" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "eef5daac82024c449fc8b0dc82bd428c" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 37677 } } }
I20240502 20:45:46.531756 24848 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ee9e40ee38ca422a8ca040a4a49b2ea7"
dest_uuid: "f926d9d9d2314c2b90437934356f66d4"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:44388
I20240502 20:45:46.531877 24848 raft_consensus.cc:595] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Received request to transfer leadership
I20240502 20:45:46.553316 24848 raft_consensus.cc:1261] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4 [term 2 FOLLOWER]: Refusing update from remote peer 85b4f683f6904c35b3c0e91799295bb3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:45:46.553972 27024 consensus_queue.cc:1035] T f258bd1f1bd14f99bed69e112cb09358 P 85b4f683f6904c35b3c0e91799295bb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:45:46.571692 26726 raft_consensus.cc:979] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4: : Instructing follower f6dd93d055454bc5965b407745c020ff to start an election
I20240502 20:45:46.571805 26726 raft_consensus.cc:1067] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Signalling peer f6dd93d055454bc5965b407745c020ff to start an election
I20240502 20:45:46.572278 26257 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "71e60972a8354e23adb03542b9935769"
dest_uuid: "f6dd93d055454bc5965b407745c020ff"
from {username='slave'} at 127.22.45.66:56845
I20240502 20:45:46.572443 26257 raft_consensus.cc:482] T 71e60972a8354e23adb03542b9935769 P f6dd93d055454bc5965b407745c020ff [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:45:46.572536 26257 raft_consensus.cc:3037] T 71e60972a8354e23adb03542b9935769 P f6dd93d055454bc5965b407745c020ff [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:45:46.573302 26257 raft_consensus.cc:504] T 71e60972a8354e23adb03542b9935769 P f6dd93d055454bc5965b407745c020ff [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "f6dd93d055454bc5965b407745c020ff" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 36617 } }
I20240502 20:45:46.574101 26257 leader_election.cc:288] T 71e60972a8354e23adb03542b9935769 P f6dd93d055454bc5965b407745c020ff [CANDIDATE]: Term 2 election: Requested vote from peers f926d9d9d2314c2b90437934356f66d4 (127.22.45.66:46041)
I20240502 20:45:46.574410 24848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71e60972a8354e23adb03542b9935769" candidate_uuid: "f6dd93d055454bc5965b407745c020ff" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f926d9d9d2314c2b90437934356f66d4"
I20240502 20:45:46.574565 24848 raft_consensus.cc:3032] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:45:46.574618 24848 raft_consensus.cc:729] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f926d9d9d2314c2b90437934356f66d4, State: Running, Role: LEADER
I20240502 20:45:46.574750 24848 consensus_queue.cc:260] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "f6dd93d055454bc5965b407745c020ff" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 36617 } }
I20240502 20:45:46.574920 24848 raft_consensus.cc:3037] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:45:46.575958 24848 raft_consensus.cc:2454] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6dd93d055454bc5965b407745c020ff in term 2.
I20240502 20:45:46.576615 26191 leader_election.cc:302] T 71e60972a8354e23adb03542b9935769 P f6dd93d055454bc5965b407745c020ff [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: f6dd93d055454bc5965b407745c020ff, f926d9d9d2314c2b90437934356f66d4; no voters:
I20240502 20:45:46.579576 27219 raft_consensus.cc:2781] T 71e60972a8354e23adb03542b9935769 P f6dd93d055454bc5965b407745c020ff [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:45:46.579780 27219 raft_consensus.cc:686] T 71e60972a8354e23adb03542b9935769 P f6dd93d055454bc5965b407745c020ff [term 2 LEADER]: Becoming Leader. State: Replica: f6dd93d055454bc5965b407745c020ff, State: Running, Role: LEADER
I20240502 20:45:46.581606 27219 consensus_queue.cc:237] T 71e60972a8354e23adb03542b9935769 P f6dd93d055454bc5965b407745c020ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "f6dd93d055454bc5965b407745c020ff" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 36617 } }
I20240502 20:45:46.583382 24696 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:45:46.585572 24568 catalog_manager.cc:5484] T 71e60972a8354e23adb03542b9935769 P f6dd93d055454bc5965b407745c020ff reported cstate change: term changed from 1 to 2, leader changed from f926d9d9d2314c2b90437934356f66d4 (127.22.45.66) to f6dd93d055454bc5965b407745c020ff (127.22.45.70). New cstate: current_term: 2 leader_uuid: "f6dd93d055454bc5965b407745c020ff" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "f6dd93d055454bc5965b407745c020ff" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 36617 } } }
I20240502 20:45:46.587121 26105 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.591231 25973 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.592691 26237 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.595712 24828 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:45:46.598408 26369 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.610208 25834 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.681774 26105 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.685984 26369 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.691254 24828 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:45:46.691362 24696 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:45:46.693802 25973 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.694933 25834 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.705632 26237 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.733822 24716 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e106fd5f2bc44c9e96bc38e2fd0dec1f"
dest_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:58256
I20240502 20:45:46.733976 24716 raft_consensus.cc:595] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Received request to transfer leadership
I20240502 20:45:46.764312 24716 raft_consensus.cc:1261] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 2 FOLLOWER]: Refusing update from remote peer bb697f05a8394524a411ece19fe608dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:45:46.764951 27116 consensus_queue.cc:1035] T 77f37012797e41cb9aafd5ab00d049ab P bb697f05a8394524a411ece19fe608dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:45:46.785048 25834 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.789258 26237 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.789394 26369 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.791250 26105 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.792502 24696 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:45:46.799340 25973 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.809337 24828 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:45:46.868489 26726 raft_consensus.cc:979] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4: : Instructing follower bb697f05a8394524a411ece19fe608dc to start an election
I20240502 20:45:46.868608 26827 raft_consensus.cc:1067] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Signalling peer bb697f05a8394524a411ece19fe608dc to start an election
I20240502 20:45:46.869248 25993 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ee9e40ee38ca422a8ca040a4a49b2ea7"
dest_uuid: "bb697f05a8394524a411ece19fe608dc"
from {username='slave'} at 127.22.45.66:49623
I20240502 20:45:46.869382 25993 raft_consensus.cc:482] T ee9e40ee38ca422a8ca040a4a49b2ea7 P bb697f05a8394524a411ece19fe608dc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:45:46.869452 25993 raft_consensus.cc:3037] T ee9e40ee38ca422a8ca040a4a49b2ea7 P bb697f05a8394524a411ece19fe608dc [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:45:46.870566 25993 raft_consensus.cc:504] T ee9e40ee38ca422a8ca040a4a49b2ea7 P bb697f05a8394524a411ece19fe608dc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "bb697f05a8394524a411ece19fe608dc" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 39501 } }
I20240502 20:45:46.870930 25993 leader_election.cc:288] T ee9e40ee38ca422a8ca040a4a49b2ea7 P bb697f05a8394524a411ece19fe608dc [CANDIDATE]: Term 2 election: Requested vote from peers f926d9d9d2314c2b90437934356f66d4 (127.22.45.66:46041)
I20240502 20:45:46.871435 24848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee9e40ee38ca422a8ca040a4a49b2ea7" candidate_uuid: "bb697f05a8394524a411ece19fe608dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f926d9d9d2314c2b90437934356f66d4"
I20240502 20:45:46.871547 24848 raft_consensus.cc:3032] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:45:46.871584 24848 raft_consensus.cc:729] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f926d9d9d2314c2b90437934356f66d4, State: Running, Role: LEADER
I20240502 20:45:46.871731 24848 consensus_queue.cc:260] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "bb697f05a8394524a411ece19fe608dc" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 39501 } }
I20240502 20:45:46.871841 24848 raft_consensus.cc:3037] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:45:46.872728 24848 raft_consensus.cc:2454] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb697f05a8394524a411ece19fe608dc in term 2.
I20240502 20:45:46.873304 25927 leader_election.cc:302] T ee9e40ee38ca422a8ca040a4a49b2ea7 P bb697f05a8394524a411ece19fe608dc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: bb697f05a8394524a411ece19fe608dc, f926d9d9d2314c2b90437934356f66d4; no voters:
I20240502 20:45:46.873560 27117 raft_consensus.cc:2781] T ee9e40ee38ca422a8ca040a4a49b2ea7 P bb697f05a8394524a411ece19fe608dc [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:45:46.873637 27117 raft_consensus.cc:686] T ee9e40ee38ca422a8ca040a4a49b2ea7 P bb697f05a8394524a411ece19fe608dc [term 2 LEADER]: Becoming Leader. State: Replica: bb697f05a8394524a411ece19fe608dc, State: Running, Role: LEADER
I20240502 20:45:46.873750 27117 consensus_queue.cc:237] T ee9e40ee38ca422a8ca040a4a49b2ea7 P bb697f05a8394524a411ece19fe608dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "bb697f05a8394524a411ece19fe608dc" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 39501 } }
I20240502 20:45:46.875373 24565 catalog_manager.cc:5484] T ee9e40ee38ca422a8ca040a4a49b2ea7 P bb697f05a8394524a411ece19fe608dc reported cstate change: term changed from 1 to 2, leader changed from f926d9d9d2314c2b90437934356f66d4 (127.22.45.66) to bb697f05a8394524a411ece19fe608dc (127.22.45.68). New cstate: current_term: 2 leader_uuid: "bb697f05a8394524a411ece19fe608dc" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "bb697f05a8394524a411ece19fe608dc" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 39501 } } }
I20240502 20:45:46.884382 24696 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:45:46.887979 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:46.889724 26369 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.889750 25834 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.899035 26105 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.900923 26237 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.901607 24828 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:45:46.933467 24716 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d661eaefcd884d1b947002588ec335ac"
dest_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:59280
I20240502 20:45:46.933598 24716 raft_consensus.cc:595] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Received request to transfer leadership
I20240502 20:45:46.984210 26369 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.985628 24696 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:45:46.988296 26237 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.988548 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:46.990176 25834 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:46.990518 24828 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:45:47.012459 26105 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:47.047382 24716 raft_consensus.cc:1261] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 2 FOLLOWER]: Refusing update from remote peer eef5daac82024c449fc8b0dc82bd428c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:45:47.047462 24848 raft_consensus.cc:1261] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4 [term 2 FOLLOWER]: Refusing update from remote peer f6dd93d055454bc5965b407745c020ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:45:47.047906 27385 consensus_queue.cc:1035] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P eef5daac82024c449fc8b0dc82bd428c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:45:47.047940 27220 consensus_queue.cc:1035] T 71e60972a8354e23adb03542b9935769 P f6dd93d055454bc5965b407745c020ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:45:47.085848 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:47.089026 26237 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:47.089058 25834 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:47.090134 26369 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:47.100796 26105 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:47.111101 24828 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:45:47.114078 24696 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:45:47.138520 24716 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3b6e517be2fe4c79bce310d8786554f4"
dest_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:59302
I20240502 20:45:47.138656 24716 raft_consensus.cc:595] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Received request to transfer leadership
I20240502 20:45:47.145295 26696 raft_consensus.cc:979] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77: : Instructing follower eef5daac82024c449fc8b0dc82bd428c to start an election
I20240502 20:45:47.145388 26696 raft_consensus.cc:1067] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Signalling peer eef5daac82024c449fc8b0dc82bd428c to start an election
I20240502 20:45:47.145792 26125 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e106fd5f2bc44c9e96bc38e2fd0dec1f"
dest_uuid: "eef5daac82024c449fc8b0dc82bd428c"
from {username='slave'} at 127.22.45.65:37859
I20240502 20:45:47.145916 26125 raft_consensus.cc:482] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:45:47.145975 26125 raft_consensus.cc:3037] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:45:47.146790 26125 raft_consensus.cc:504] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "eef5daac82024c449fc8b0dc82bd428c" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 37677 } }
I20240502 20:45:47.147059 26125 leader_election.cc:288] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c [CANDIDATE]: Term 2 election: Requested vote from peers 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65:36905)
I20240502 20:45:47.147382 24716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e106fd5f2bc44c9e96bc38e2fd0dec1f" candidate_uuid: "eef5daac82024c449fc8b0dc82bd428c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77"
I20240502 20:45:47.147487 24716 raft_consensus.cc:3032] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:45:47.147531 24716 raft_consensus.cc:729] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1c52358e8e2f4ffe992cafe0f1a95a77, State: Running, Role: LEADER
I20240502 20:45:47.147665 24716 consensus_queue.cc:260] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "eef5daac82024c449fc8b0dc82bd428c" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 37677 } }
I20240502 20:45:47.147783 24716 raft_consensus.cc:3037] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:45:47.148488 24716 raft_consensus.cc:2454] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eef5daac82024c449fc8b0dc82bd428c in term 2.
I20240502 20:45:47.148738 26061 leader_election.cc:302] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1c52358e8e2f4ffe992cafe0f1a95a77, eef5daac82024c449fc8b0dc82bd428c; no voters:
I20240502 20:45:47.148885 27385 raft_consensus.cc:2781] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:45:47.148967 27385 raft_consensus.cc:686] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c [term 2 LEADER]: Becoming Leader. State: Replica: eef5daac82024c449fc8b0dc82bd428c, State: Running, Role: LEADER
I20240502 20:45:47.149087 27385 consensus_queue.cc:237] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "eef5daac82024c449fc8b0dc82bd428c" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 37677 } }
I20240502 20:45:47.150727 24568 catalog_manager.cc:5484] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c reported cstate change: term changed from 1 to 2, leader changed from 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65) to eef5daac82024c449fc8b0dc82bd428c (127.22.45.69). New cstate: current_term: 2 leader_uuid: "eef5daac82024c449fc8b0dc82bd428c" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "eef5daac82024c449fc8b0dc82bd428c" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 37677 } } }
I20240502 20:45:47.186959 26369 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:47.188159 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:47.189996 24696 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:45:47.194876 25834 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:47.195878 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:47.208567 24828 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:45:47.219151 26237 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:47.262025 27428 raft_consensus.cc:979] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77: : Instructing follower a984b4b59cb6429290bc907a860d50f8 to start an election
I20240502 20:45:47.262136 27428 raft_consensus.cc:1067] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Signalling peer a984b4b59cb6429290bc907a860d50f8 to start an election
I20240502 20:45:47.263232 25854 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d661eaefcd884d1b947002588ec335ac"
dest_uuid: "a984b4b59cb6429290bc907a860d50f8"
from {username='slave'} at 127.22.45.65:39149
I20240502 20:45:47.263427 25854 raft_consensus.cc:482] T d661eaefcd884d1b947002588ec335ac P a984b4b59cb6429290bc907a860d50f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:45:47.263486 25854 raft_consensus.cc:3037] T d661eaefcd884d1b947002588ec335ac P a984b4b59cb6429290bc907a860d50f8 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:45:47.264493 25854 raft_consensus.cc:504] T d661eaefcd884d1b947002588ec335ac P a984b4b59cb6429290bc907a860d50f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "a984b4b59cb6429290bc907a860d50f8" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44809 } }
I20240502 20:45:47.265221 25854 leader_election.cc:288] T d661eaefcd884d1b947002588ec335ac P a984b4b59cb6429290bc907a860d50f8 [CANDIDATE]: Term 2 election: Requested vote from peers 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65:36905)
I20240502 20:45:47.265600 24716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d661eaefcd884d1b947002588ec335ac" candidate_uuid: "a984b4b59cb6429290bc907a860d50f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77"
I20240502 20:45:47.265715 24716 raft_consensus.cc:3032] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:45:47.265758 24716 raft_consensus.cc:729] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1c52358e8e2f4ffe992cafe0f1a95a77, State: Running, Role: LEADER
I20240502 20:45:47.265900 24716 consensus_queue.cc:260] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "a984b4b59cb6429290bc907a860d50f8" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44809 } }
I20240502 20:45:47.266059 24716 raft_consensus.cc:3037] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:45:47.267026 24716 raft_consensus.cc:2454] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a984b4b59cb6429290bc907a860d50f8 in term 2.
I20240502 20:45:47.267340 25789 leader_election.cc:302] T d661eaefcd884d1b947002588ec335ac P a984b4b59cb6429290bc907a860d50f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1c52358e8e2f4ffe992cafe0f1a95a77, a984b4b59cb6429290bc907a860d50f8; no voters:
I20240502 20:45:47.267606 27472 raft_consensus.cc:2781] T d661eaefcd884d1b947002588ec335ac P a984b4b59cb6429290bc907a860d50f8 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:45:47.267755 27472 raft_consensus.cc:686] T d661eaefcd884d1b947002588ec335ac P a984b4b59cb6429290bc907a860d50f8 [term 2 LEADER]: Becoming Leader. State: Replica: a984b4b59cb6429290bc907a860d50f8, State: Running, Role: LEADER
I20240502 20:45:47.268086 27472 consensus_queue.cc:237] T d661eaefcd884d1b947002588ec335ac P a984b4b59cb6429290bc907a860d50f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "a984b4b59cb6429290bc907a860d50f8" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44809 } }
I20240502 20:45:47.270143 24571 catalog_manager.cc:5484] T d661eaefcd884d1b947002588ec335ac P a984b4b59cb6429290bc907a860d50f8 reported cstate change: term changed from 1 to 2, leader changed from 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65) to a984b4b59cb6429290bc907a860d50f8 (127.22.45.67). New cstate: current_term: 2 leader_uuid: "a984b4b59cb6429290bc907a860d50f8" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "a984b4b59cb6429290bc907a860d50f8" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44809 } } }
I20240502 20:45:47.291651 24848 raft_consensus.cc:1261] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4 [term 2 FOLLOWER]: Refusing update from remote peer bb697f05a8394524a411ece19fe608dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:45:47.292131 27116 consensus_queue.cc:1035] T ee9e40ee38ca422a8ca040a4a49b2ea7 P bb697f05a8394524a411ece19fe608dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:45:47.299485 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:47.301102 24696 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:45:47.301354 25834 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:47.301645 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:47.302634 24828 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:45:47.309103 26237 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:47.311209 26369 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:47.344167 24848 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "61dc146b84774046980dc8fdbf8e632f"
dest_uuid: "f926d9d9d2314c2b90437934356f66d4"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:59030
I20240502 20:45:47.344305 24848 raft_consensus.cc:595] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Received request to transfer leadership
I20240502 20:45:47.369220 27428 raft_consensus.cc:979] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77: : Instructing follower 85b4f683f6904c35b3c0e91799295bb3 to start an election
I20240502 20:45:47.369344 27428 raft_consensus.cc:1067] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Signalling peer 85b4f683f6904c35b3c0e91799295bb3 to start an election
I20240502 20:45:47.369801 26389 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3b6e517be2fe4c79bce310d8786554f4"
dest_uuid: "85b4f683f6904c35b3c0e91799295bb3"
from {username='slave'} at 127.22.45.65:44107
I20240502 20:45:47.369927 26389 raft_consensus.cc:482] T 3b6e517be2fe4c79bce310d8786554f4 P 85b4f683f6904c35b3c0e91799295bb3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:45:47.369987 26389 raft_consensus.cc:3037] T 3b6e517be2fe4c79bce310d8786554f4 P 85b4f683f6904c35b3c0e91799295bb3 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:45:47.371090 26389 raft_consensus.cc:504] T 3b6e517be2fe4c79bce310d8786554f4 P 85b4f683f6904c35b3c0e91799295bb3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "85b4f683f6904c35b3c0e91799295bb3" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 39125 } }
I20240502 20:45:47.371474 26389 leader_election.cc:288] T 3b6e517be2fe4c79bce310d8786554f4 P 85b4f683f6904c35b3c0e91799295bb3 [CANDIDATE]: Term 2 election: Requested vote from peers 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65:36905)
I20240502 20:45:47.371870 24716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b6e517be2fe4c79bce310d8786554f4" candidate_uuid: "85b4f683f6904c35b3c0e91799295bb3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77"
I20240502 20:45:47.371994 24716 raft_consensus.cc:3032] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:45:47.372040 24716 raft_consensus.cc:729] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1c52358e8e2f4ffe992cafe0f1a95a77, State: Running, Role: LEADER
I20240502 20:45:47.372184 24716 consensus_queue.cc:260] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "85b4f683f6904c35b3c0e91799295bb3" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 39125 } }
I20240502 20:45:47.372341 24716 raft_consensus.cc:3037] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:45:47.373163 24716 raft_consensus.cc:2454] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85b4f683f6904c35b3c0e91799295bb3 in term 2.
I20240502 20:45:47.373512 26325 leader_election.cc:302] T 3b6e517be2fe4c79bce310d8786554f4 P 85b4f683f6904c35b3c0e91799295bb3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1c52358e8e2f4ffe992cafe0f1a95a77, 85b4f683f6904c35b3c0e91799295bb3; no voters:
I20240502 20:45:47.373737 26826 raft_consensus.cc:2781] T 3b6e517be2fe4c79bce310d8786554f4 P 85b4f683f6904c35b3c0e91799295bb3 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:45:47.373888 26826 raft_consensus.cc:686] T 3b6e517be2fe4c79bce310d8786554f4 P 85b4f683f6904c35b3c0e91799295bb3 [term 2 LEADER]: Becoming Leader. State: Replica: 85b4f683f6904c35b3c0e91799295bb3, State: Running, Role: LEADER
I20240502 20:45:47.374018 26826 consensus_queue.cc:237] T 3b6e517be2fe4c79bce310d8786554f4 P 85b4f683f6904c35b3c0e91799295bb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "85b4f683f6904c35b3c0e91799295bb3" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 39125 } }
I20240502 20:45:47.375888 24565 catalog_manager.cc:5484] T 3b6e517be2fe4c79bce310d8786554f4 P 85b4f683f6904c35b3c0e91799295bb3 reported cstate change: term changed from 1 to 2, leader changed from 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65) to 85b4f683f6904c35b3c0e91799295bb3 (127.22.45.71). New cstate: current_term: 2 leader_uuid: "85b4f683f6904c35b3c0e91799295bb3" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "85b4f683f6904c35b3c0e91799295bb3" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 39125 } } }
I20240502 20:45:47.391968 26369 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:47.392987 25834 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:47.394490 24696 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:47.401433 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:47.406610 24828 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:45:47.414630 26237 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:47.423148 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:47.665958 24716 raft_consensus.cc:1261] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 2 FOLLOWER]: Refusing update from remote peer eef5daac82024c449fc8b0dc82bd428c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:45:47.666399 27385 consensus_queue.cc:1035] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:45:47.700726 26369 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:47.701241 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:47.701400 24696 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:47.706775 25834 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:47.711879 24716 raft_consensus.cc:1261] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 2 FOLLOWER]: Refusing update from remote peer a984b4b59cb6429290bc907a860d50f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:45:47.712436 27472 consensus_queue.cc:1035] T d661eaefcd884d1b947002588ec335ac P a984b4b59cb6429290bc907a860d50f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:45:47.718317 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:47.722213 26237 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:47.727813 24828 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:45:47.772126 26389 consensus_queue.cc:237] T f258bd1f1bd14f99bed69e112cb09358 P 85b4f683f6904c35b3c0e91799295bb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "85b4f683f6904c35b3c0e91799295bb3" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 39125 } }
I20240502 20:45:47.772969 26826 raft_consensus.cc:2932] T f258bd1f1bd14f99bed69e112cb09358 P 85b4f683f6904c35b3c0e91799295bb3 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER f926d9d9d2314c2b90437934356f66d4 (127.22.45.66) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "85b4f683f6904c35b3c0e91799295bb3" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 39125 } } }
I20240502 20:45:47.775388 24571 catalog_manager.cc:5484] T f258bd1f1bd14f99bed69e112cb09358 P 85b4f683f6904c35b3c0e91799295bb3 reported cstate change: config changed from index 4 to 6, VOTER f926d9d9d2314c2b90437934356f66d4 (127.22.45.66) evicted. New cstate: current_term: 2 leader_uuid: "85b4f683f6904c35b3c0e91799295bb3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "85b4f683f6904c35b3c0e91799295bb3" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 39125 } } }
I20240502 20:45:47.778100 24828 tablet_service.cc:1514] Processing DeleteTablet for tablet f258bd1f1bd14f99bed69e112cb09358 with delete_type TABLET_DATA_TOMBSTONED (TS f926d9d9d2314c2b90437934356f66d4 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:35518
I20240502 20:45:47.778616 27602 tablet_replica.cc:330] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4: stopping tablet replica
I20240502 20:45:47.778919 27602 raft_consensus.cc:2229] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 20:45:47.779071 27602 raft_consensus.cc:2258] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 20:45:47.780028 27602 ts_tablet_manager.cc:1875] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:45:47.782286 27602 ts_tablet_manager.cc:1888] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 20:45:47.782371 27602 log.cc:1196] T f258bd1f1bd14f99bed69e112cb09358 P f926d9d9d2314c2b90437934356f66d4: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/f258bd1f1bd14f99bed69e112cb09358
I20240502 20:45:47.783053 24558 catalog_manager.cc:4830] TS f926d9d9d2314c2b90437934356f66d4 (127.22.45.66:46041): tablet f258bd1f1bd14f99bed69e112cb09358 (table rebalance_test_table_1 [id=2214dd294ae048a4926821c50ed5e793]) successfully deleted
I20240502 20:45:47.834213 25834 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:47.838949 24828 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:45:47.839689 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:47.845010 24696 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:47.855674 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:47.856590 26237 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:45:47.857151 26369 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:47.874974 27635 raft_consensus.cc:979] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4: : Instructing follower f6dd93d055454bc5965b407745c020ff to start an election
I20240502 20:45:47.875066 27635 raft_consensus.cc:1067] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Signalling peer f6dd93d055454bc5965b407745c020ff to start an election
I20240502 20:45:47.875460 26257 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "61dc146b84774046980dc8fdbf8e632f"
dest_uuid: "f6dd93d055454bc5965b407745c020ff"
from {username='slave'} at 127.22.45.66:56845
I20240502 20:45:47.875597 26257 raft_consensus.cc:482] T 61dc146b84774046980dc8fdbf8e632f P f6dd93d055454bc5965b407745c020ff [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:45:47.875651 26257 raft_consensus.cc:3037] T 61dc146b84774046980dc8fdbf8e632f P f6dd93d055454bc5965b407745c020ff [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:45:47.876659 26257 raft_consensus.cc:504] T 61dc146b84774046980dc8fdbf8e632f P f6dd93d055454bc5965b407745c020ff [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "f6dd93d055454bc5965b407745c020ff" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 36617 } }
I20240502 20:45:47.876963 26257 leader_election.cc:288] T 61dc146b84774046980dc8fdbf8e632f P f6dd93d055454bc5965b407745c020ff [CANDIDATE]: Term 2 election: Requested vote from peers f926d9d9d2314c2b90437934356f66d4 (127.22.45.66:46041)
I20240502 20:45:47.877377 24848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61dc146b84774046980dc8fdbf8e632f" candidate_uuid: "f6dd93d055454bc5965b407745c020ff" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f926d9d9d2314c2b90437934356f66d4"
I20240502 20:45:47.877501 24848 raft_consensus.cc:3032] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:45:47.877543 24848 raft_consensus.cc:729] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f926d9d9d2314c2b90437934356f66d4, State: Running, Role: LEADER
I20240502 20:45:47.877681 24848 consensus_queue.cc:260] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "f6dd93d055454bc5965b407745c020ff" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 36617 } }
I20240502 20:45:47.877820 24848 raft_consensus.cc:3037] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:45:47.878757 24848 raft_consensus.cc:2454] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6dd93d055454bc5965b407745c020ff in term 2.
I20240502 20:45:47.879070 26191 leader_election.cc:302] T 61dc146b84774046980dc8fdbf8e632f P f6dd93d055454bc5965b407745c020ff [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: f6dd93d055454bc5965b407745c020ff, f926d9d9d2314c2b90437934356f66d4; no voters:
I20240502 20:45:47.879316 27537 raft_consensus.cc:2781] T 61dc146b84774046980dc8fdbf8e632f P f6dd93d055454bc5965b407745c020ff [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:45:47.879462 27537 raft_consensus.cc:686] T 61dc146b84774046980dc8fdbf8e632f P f6dd93d055454bc5965b407745c020ff [term 2 LEADER]: Becoming Leader. State: Replica: f6dd93d055454bc5965b407745c020ff, State: Running, Role: LEADER
I20240502 20:45:47.879576 27537 consensus_queue.cc:237] T 61dc146b84774046980dc8fdbf8e632f P f6dd93d055454bc5965b407745c020ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "f6dd93d055454bc5965b407745c020ff" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 36617 } }
I20240502 20:45:47.881047 24568 catalog_manager.cc:5484] T 61dc146b84774046980dc8fdbf8e632f P f6dd93d055454bc5965b407745c020ff reported cstate change: term changed from 1 to 2, leader changed from f926d9d9d2314c2b90437934356f66d4 (127.22.45.66) to f6dd93d055454bc5965b407745c020ff (127.22.45.70). New cstate: current_term: 2 leader_uuid: "f6dd93d055454bc5965b407745c020ff" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "f6dd93d055454bc5965b407745c020ff" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 36617 } } }
I20240502 20:45:47.900152 25993 consensus_queue.cc:237] T 77f37012797e41cb9aafd5ab00d049ab P bb697f05a8394524a411ece19fe608dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bb697f05a8394524a411ece19fe608dc" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 39501 } }
I20240502 20:45:47.900808 27117 raft_consensus.cc:2932] T 77f37012797e41cb9aafd5ab00d049ab P bb697f05a8394524a411ece19fe608dc [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bb697f05a8394524a411ece19fe608dc" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 39501 } } }
I20240502 20:45:47.902818 24568 catalog_manager.cc:5484] T 77f37012797e41cb9aafd5ab00d049ab P bb697f05a8394524a411ece19fe608dc reported cstate change: config changed from index 4 to 6, VOTER 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65) evicted. New cstate: current_term: 2 leader_uuid: "bb697f05a8394524a411ece19fe608dc" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bb697f05a8394524a411ece19fe608dc" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 39501 } } }
I20240502 20:45:47.905089 24696 tablet_service.cc:1514] Processing DeleteTablet for tablet 77f37012797e41cb9aafd5ab00d049ab with delete_type TABLET_DATA_TOMBSTONED (TS 1c52358e8e2f4ffe992cafe0f1a95a77 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:34622
I20240502 20:45:47.905562 27661 tablet_replica.cc:330] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77: stopping tablet replica
I20240502 20:45:47.905820 27661 raft_consensus.cc:2229] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 20:45:47.905962 27661 raft_consensus.cc:2258] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 20:45:47.906842 27661 ts_tablet_manager.cc:1875] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:45:47.908826 27661 ts_tablet_manager.cc:1888] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 20:45:47.908908 27661 log.cc:1196] T 77f37012797e41cb9aafd5ab00d049ab P 1c52358e8e2f4ffe992cafe0f1a95a77: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/77f37012797e41cb9aafd5ab00d049ab
I20240502 20:45:47.909487 24560 catalog_manager.cc:4830] TS 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65:36905): tablet 77f37012797e41cb9aafd5ab00d049ab (table rebalance_test_table_0 [id=f8e7a465361144c9a131d92402ce77fb]) successfully deleted
I20240502 20:45:47.959537 24716 raft_consensus.cc:1261] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 2 FOLLOWER]: Refusing update from remote peer 85b4f683f6904c35b3c0e91799295bb3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:45:47.959827 24696 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:47.960505 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:47.961238 26369 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:47.961174 26826 consensus_queue.cc:1035] T 3b6e517be2fe4c79bce310d8786554f4 P 85b4f683f6904c35b3c0e91799295bb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:45:47.963687 26237 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:47.966732 25834 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:47.971923 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:47.977418 24828 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:48.025523 26125 consensus_queue.cc:237] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P eef5daac82024c449fc8b0dc82bd428c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "eef5daac82024c449fc8b0dc82bd428c" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 37677 } }
I20240502 20:45:48.026136 27554 raft_consensus.cc:2932] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P eef5daac82024c449fc8b0dc82bd428c [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "eef5daac82024c449fc8b0dc82bd428c" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 37677 } } }
I20240502 20:45:48.028182 24567 catalog_manager.cc:5484] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P eef5daac82024c449fc8b0dc82bd428c reported cstate change: config changed from index 4 to 6, VOTER 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65) evicted. New cstate: current_term: 2 leader_uuid: "eef5daac82024c449fc8b0dc82bd428c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "eef5daac82024c449fc8b0dc82bd428c" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 37677 } } }
I20240502 20:45:48.030491 24696 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ed23be7cabc44b4ad4db0ee4f2e5392 with delete_type TABLET_DATA_TOMBSTONED (TS 1c52358e8e2f4ffe992cafe0f1a95a77 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:34622
I20240502 20:45:48.030704 27661 tablet_replica.cc:330] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77: stopping tablet replica
I20240502 20:45:48.030822 27661 raft_consensus.cc:2229] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 20:45:48.030925 27661 raft_consensus.cc:2258] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 20:45:48.031651 27661 ts_tablet_manager.cc:1875] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:45:48.033720 27661 ts_tablet_manager.cc:1888] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 20:45:48.033803 27661 log.cc:1196] T 7ed23be7cabc44b4ad4db0ee4f2e5392 P 1c52358e8e2f4ffe992cafe0f1a95a77: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/7ed23be7cabc44b4ad4db0ee4f2e5392
I20240502 20:45:48.034288 24560 catalog_manager.cc:4830] TS 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65:36905): tablet 7ed23be7cabc44b4ad4db0ee4f2e5392 (table rebalance_test_table_3 [id=8c45f90c8384402aaeaf9e17e4452dd9]) successfully deleted
I20240502 20:45:48.083575 24696 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:48.084766 26369 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.085014 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.086710 26237 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.088691 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.108177 25834 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.111579 24828 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:48.143956 26257 consensus_queue.cc:237] T 71e60972a8354e23adb03542b9935769 P f6dd93d055454bc5965b407745c020ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f6dd93d055454bc5965b407745c020ff" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 36617 } }
I20240502 20:45:48.144662 27537 raft_consensus.cc:2932] T 71e60972a8354e23adb03542b9935769 P f6dd93d055454bc5965b407745c020ff [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER f926d9d9d2314c2b90437934356f66d4 (127.22.45.66) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f6dd93d055454bc5965b407745c020ff" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 36617 } } }
I20240502 20:45:48.146711 24564 catalog_manager.cc:5484] T 71e60972a8354e23adb03542b9935769 P f6dd93d055454bc5965b407745c020ff reported cstate change: config changed from index 4 to 6, VOTER f926d9d9d2314c2b90437934356f66d4 (127.22.45.66) evicted. New cstate: current_term: 2 leader_uuid: "f6dd93d055454bc5965b407745c020ff" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f6dd93d055454bc5965b407745c020ff" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 36617 } } }
I20240502 20:45:48.149101 24828 tablet_service.cc:1514] Processing DeleteTablet for tablet 71e60972a8354e23adb03542b9935769 with delete_type TABLET_DATA_TOMBSTONED (TS f926d9d9d2314c2b90437934356f66d4 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:35518
I20240502 20:45:48.149343 27602 tablet_replica.cc:330] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4: stopping tablet replica
I20240502 20:45:48.149488 27602 raft_consensus.cc:2229] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 20:45:48.149606 27602 raft_consensus.cc:2258] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 20:45:48.150120 27602 ts_tablet_manager.cc:1875] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:45:48.152186 27602 ts_tablet_manager.cc:1888] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 20:45:48.152263 27602 log.cc:1196] T 71e60972a8354e23adb03542b9935769 P f926d9d9d2314c2b90437934356f66d4: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/71e60972a8354e23adb03542b9935769
I20240502 20:45:48.152715 24558 catalog_manager.cc:4830] TS f926d9d9d2314c2b90437934356f66d4 (127.22.45.66:46041): tablet 71e60972a8354e23adb03542b9935769 (table rebalance_test_table_3 [id=8c45f90c8384402aaeaf9e17e4452dd9]) successfully deleted
I20240502 20:45:48.197806 25834 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.199167 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.202452 24696 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:48.207701 26369 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.208086 26237 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.212312 24828 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:48.213719 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.256547 24848 raft_consensus.cc:1261] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4 [term 2 FOLLOWER]: Refusing update from remote peer f6dd93d055454bc5965b407745c020ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:45:48.256944 27537 consensus_queue.cc:1035] T 61dc146b84774046980dc8fdbf8e632f P f6dd93d055454bc5965b407745c020ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:45:48.260486 25993 consensus_queue.cc:237] T ee9e40ee38ca422a8ca040a4a49b2ea7 P bb697f05a8394524a411ece19fe608dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bb697f05a8394524a411ece19fe608dc" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 39501 } }
I20240502 20:45:48.261031 27655 raft_consensus.cc:2932] T ee9e40ee38ca422a8ca040a4a49b2ea7 P bb697f05a8394524a411ece19fe608dc [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER f926d9d9d2314c2b90437934356f66d4 (127.22.45.66) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bb697f05a8394524a411ece19fe608dc" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 39501 } } }
I20240502 20:45:48.263175 24568 catalog_manager.cc:5484] T ee9e40ee38ca422a8ca040a4a49b2ea7 P bb697f05a8394524a411ece19fe608dc reported cstate change: config changed from index 4 to 6, VOTER f926d9d9d2314c2b90437934356f66d4 (127.22.45.66) evicted. New cstate: current_term: 2 leader_uuid: "bb697f05a8394524a411ece19fe608dc" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bb697f05a8394524a411ece19fe608dc" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 39501 } } }
I20240502 20:45:48.265391 24828 tablet_service.cc:1514] Processing DeleteTablet for tablet ee9e40ee38ca422a8ca040a4a49b2ea7 with delete_type TABLET_DATA_TOMBSTONED (TS f926d9d9d2314c2b90437934356f66d4 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:35518
I20240502 20:45:48.265579 27602 tablet_replica.cc:330] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4: stopping tablet replica
I20240502 20:45:48.265703 27602 raft_consensus.cc:2229] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 20:45:48.265794 27602 raft_consensus.cc:2258] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 20:45:48.266316 27602 ts_tablet_manager.cc:1875] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:45:48.267922 27602 ts_tablet_manager.cc:1888] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 20:45:48.267982 27602 log.cc:1196] T ee9e40ee38ca422a8ca040a4a49b2ea7 P f926d9d9d2314c2b90437934356f66d4: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/ee9e40ee38ca422a8ca040a4a49b2ea7
I20240502 20:45:48.268383 24558 catalog_manager.cc:4830] TS f926d9d9d2314c2b90437934356f66d4 (127.22.45.66:46041): tablet ee9e40ee38ca422a8ca040a4a49b2ea7 (table rebalance_test_table_2 [id=dbf6f06f0516420ab4d3fb51341aa469]) successfully deleted
I20240502 20:45:48.323315 25834 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.324023 26369 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.326265 24696 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:48.326360 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.328042 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.337627 26237 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.353853 24828 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:48.385159 26125 consensus_queue.cc:237] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "eef5daac82024c449fc8b0dc82bd428c" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 37677 } }
I20240502 20:45:48.385720 27554 raft_consensus.cc:2932] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "eef5daac82024c449fc8b0dc82bd428c" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 37677 } } }
I20240502 20:45:48.387784 24571 catalog_manager.cc:5484] T e106fd5f2bc44c9e96bc38e2fd0dec1f P eef5daac82024c449fc8b0dc82bd428c reported cstate change: config changed from index 4 to 6, VOTER 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65) evicted. New cstate: current_term: 2 leader_uuid: "eef5daac82024c449fc8b0dc82bd428c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "eef5daac82024c449fc8b0dc82bd428c" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 37677 } } }
I20240502 20:45:48.390221 24696 tablet_service.cc:1514] Processing DeleteTablet for tablet e106fd5f2bc44c9e96bc38e2fd0dec1f with delete_type TABLET_DATA_TOMBSTONED (TS 1c52358e8e2f4ffe992cafe0f1a95a77 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:34622
I20240502 20:45:48.390511 27661 tablet_replica.cc:330] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77: stopping tablet replica
I20240502 20:45:48.390641 27661 raft_consensus.cc:2229] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 20:45:48.390746 27661 raft_consensus.cc:2258] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 20:45:48.391353 27661 ts_tablet_manager.cc:1875] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:45:48.393147 27661 ts_tablet_manager.cc:1888] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 20:45:48.393213 27661 log.cc:1196] T e106fd5f2bc44c9e96bc38e2fd0dec1f P 1c52358e8e2f4ffe992cafe0f1a95a77: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/e106fd5f2bc44c9e96bc38e2fd0dec1f
I20240502 20:45:48.393658 24560 catalog_manager.cc:4830] TS 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65:36905): tablet e106fd5f2bc44c9e96bc38e2fd0dec1f (table rebalance_test_table_2 [id=dbf6f06f0516420ab4d3fb51341aa469]) successfully deleted
I20240502 20:45:48.445472 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.445802 25834 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.446055 24696 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:48.446255 26369 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.463928 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.465633 26237 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.473224 24828 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:48.504666 25854 consensus_queue.cc:237] T d661eaefcd884d1b947002588ec335ac P a984b4b59cb6429290bc907a860d50f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a984b4b59cb6429290bc907a860d50f8" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44809 } }
I20240502 20:45:48.505252 27824 raft_consensus.cc:2932] T d661eaefcd884d1b947002588ec335ac P a984b4b59cb6429290bc907a860d50f8 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a984b4b59cb6429290bc907a860d50f8" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44809 } } }
I20240502 20:45:48.507707 24571 catalog_manager.cc:5484] T d661eaefcd884d1b947002588ec335ac P a984b4b59cb6429290bc907a860d50f8 reported cstate change: config changed from index 4 to 6, VOTER 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65) evicted. New cstate: current_term: 2 leader_uuid: "a984b4b59cb6429290bc907a860d50f8" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a984b4b59cb6429290bc907a860d50f8" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44809 } } }
I20240502 20:45:48.509932 24696 tablet_service.cc:1514] Processing DeleteTablet for tablet d661eaefcd884d1b947002588ec335ac with delete_type TABLET_DATA_TOMBSTONED (TS 1c52358e8e2f4ffe992cafe0f1a95a77 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:34622
I20240502 20:45:48.510149 27661 tablet_replica.cc:330] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77: stopping tablet replica
I20240502 20:45:48.510272 27661 raft_consensus.cc:2229] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 20:45:48.510370 27661 raft_consensus.cc:2258] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 20:45:48.510967 27661 ts_tablet_manager.cc:1875] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:45:48.512864 27661 ts_tablet_manager.cc:1888] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 20:45:48.512936 27661 log.cc:1196] T d661eaefcd884d1b947002588ec335ac P 1c52358e8e2f4ffe992cafe0f1a95a77: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/d661eaefcd884d1b947002588ec335ac
I20240502 20:45:48.513353 24560 catalog_manager.cc:4830] TS 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65:36905): tablet d661eaefcd884d1b947002588ec335ac (table rebalance_test_table_0 [id=f8e7a465361144c9a131d92402ce77fb]) successfully deleted
I20240502 20:45:48.559617 24696 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:48.562176 25834 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.563519 26369 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.573040 26237 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.583781 24828 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:48.584381 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.584635 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.624634 26389 consensus_queue.cc:237] T 3b6e517be2fe4c79bce310d8786554f4 P 85b4f683f6904c35b3c0e91799295bb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "85b4f683f6904c35b3c0e91799295bb3" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 39125 } }
I20240502 20:45:48.625253 27520 raft_consensus.cc:2932] T 3b6e517be2fe4c79bce310d8786554f4 P 85b4f683f6904c35b3c0e91799295bb3 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "85b4f683f6904c35b3c0e91799295bb3" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 39125 } } }
I20240502 20:45:48.627665 24572 catalog_manager.cc:5484] T 3b6e517be2fe4c79bce310d8786554f4 P 85b4f683f6904c35b3c0e91799295bb3 reported cstate change: config changed from index 4 to 6, VOTER 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65) evicted. New cstate: current_term: 2 leader_uuid: "85b4f683f6904c35b3c0e91799295bb3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "85b4f683f6904c35b3c0e91799295bb3" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 39125 } } }
I20240502 20:45:48.629993 24696 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b6e517be2fe4c79bce310d8786554f4 with delete_type TABLET_DATA_TOMBSTONED (TS 1c52358e8e2f4ffe992cafe0f1a95a77 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:34622
I20240502 20:45:48.630229 27661 tablet_replica.cc:330] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77: stopping tablet replica
I20240502 20:45:48.630362 27661 raft_consensus.cc:2229] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 20:45:48.630494 27661 raft_consensus.cc:2258] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 20:45:48.631107 27661 ts_tablet_manager.cc:1875] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:45:48.633103 27661 ts_tablet_manager.cc:1888] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 20:45:48.633176 27661 log.cc:1196] T 3b6e517be2fe4c79bce310d8786554f4 P 1c52358e8e2f4ffe992cafe0f1a95a77: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/3b6e517be2fe4c79bce310d8786554f4
I20240502 20:45:48.633656 24560 catalog_manager.cc:4830] TS 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65:36905): tablet 3b6e517be2fe4c79bce310d8786554f4 (table rebalance_test_table_4 [id=493ba88faedf41dcb0c39446bb42569d]) successfully deleted
I20240502 20:45:48.681780 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.683128 24696 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:48.684276 25834 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.687675 26369 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.690662 26237 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.698091 24828 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:48.711125 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:48.744913 26257 consensus_queue.cc:237] T 61dc146b84774046980dc8fdbf8e632f P f6dd93d055454bc5965b407745c020ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f6dd93d055454bc5965b407745c020ff" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 36617 } }
I20240502 20:45:48.745507 27637 raft_consensus.cc:2932] T 61dc146b84774046980dc8fdbf8e632f P f6dd93d055454bc5965b407745c020ff [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER f926d9d9d2314c2b90437934356f66d4 (127.22.45.66) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f6dd93d055454bc5965b407745c020ff" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 36617 } } }
I20240502 20:45:48.747562 24567 catalog_manager.cc:5484] T 61dc146b84774046980dc8fdbf8e632f P f6dd93d055454bc5965b407745c020ff reported cstate change: config changed from index 4 to 6, VOTER f926d9d9d2314c2b90437934356f66d4 (127.22.45.66) evicted. New cstate: current_term: 2 leader_uuid: "f6dd93d055454bc5965b407745c020ff" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f6dd93d055454bc5965b407745c020ff" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 36617 } } }
I20240502 20:45:48.749960 24828 tablet_service.cc:1514] Processing DeleteTablet for tablet 61dc146b84774046980dc8fdbf8e632f with delete_type TABLET_DATA_TOMBSTONED (TS f926d9d9d2314c2b90437934356f66d4 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:35518
I20240502 20:45:48.750201 27602 tablet_replica.cc:330] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4: stopping tablet replica
I20240502 20:45:48.750370 27602 raft_consensus.cc:2229] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 20:45:48.750523 27602 raft_consensus.cc:2258] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 20:45:48.751137 27602 ts_tablet_manager.cc:1875] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:45:48.752895 27602 ts_tablet_manager.cc:1888] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 20:45:48.752969 27602 log.cc:1196] T 61dc146b84774046980dc8fdbf8e632f P f926d9d9d2314c2b90437934356f66d4: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/61dc146b84774046980dc8fdbf8e632f
I20240502 20:45:48.753458 24558 catalog_manager.cc:4830] TS f926d9d9d2314c2b90437934356f66d4 (127.22.45.66:46041): tablet 61dc146b84774046980dc8fdbf8e632f (table rebalance_test_table_1 [id=2214dd294ae048a4926821c50ed5e793]) successfully deleted
I20240502 20:45:49.047343 24696 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:49.047355 26369 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:49.047936 26237 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:49.048590 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:49.055788 24828 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:49.056541 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:49.076653 25834 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:49.150348 26369 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:49.150640 25834 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:49.151458 24696 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:49.155066 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:49.155459 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:49.155664 24828 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:49.156834 26237 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:49.385848 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:49.386565 26369 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:49.386715 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:49.386991 25834 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:49.387269 24696 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:49.387651 26237 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:49.391855 24828 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:49.505820 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:49.509058 25834 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:49.510564 26237 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:49.511092 24696 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:49.511739 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:49.523275 24828 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:49.531311 26369 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:49.618782 24696 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:49.623488 26237 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:49.625502 26369 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:49.628012 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:49.629256 25834 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:49.634634 24828 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:49.650270 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
fe9820ffe81d4edb865d80b0d7faae0c | 127.22.45.126:34337 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
allow_unsafe_replication_factor | true | runtime,unsafe | all 1 server(s) checked
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.22.45.84:37045 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
1c52358e8e2f4ffe992cafe0f1a95a77 | 127.22.45.65:36905 | HEALTHY | <none> | 3 | 0
85b4f683f6904c35b3c0e91799295bb3 | 127.22.45.71:39125 | HEALTHY | <none> | 2 | 0
a984b4b59cb6429290bc907a860d50f8 | 127.22.45.67:44809 | HEALTHY | <none> | 2 | 0
bb697f05a8394524a411ece19fe608dc | 127.22.45.68:39501 | HEALTHY | <none> | 2 | 0
eef5daac82024c449fc8b0dc82bd428c | 127.22.45.69:37677 | HEALTHY | <none> | 2 | 0
f6dd93d055454bc5965b407745c020ff | 127.22.45.70:36617 | HEALTHY | <none> | 2 | 0
f926d9d9d2314c2b90437934356f66d4 | 127.22.45.66:46041 | HEALTHY | <none> | 3 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 7
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 7 server(s) checked
local_ip_for_outbound_sockets | 127.22.45.65 | experimental | 127.22.45.65:36905
local_ip_for_outbound_sockets | 127.22.45.66 | experimental | 127.22.45.66:46041
local_ip_for_outbound_sockets | 127.22.45.67 | experimental | 127.22.45.67:44809
local_ip_for_outbound_sockets | 127.22.45.68 | experimental | 127.22.45.68:39501
local_ip_for_outbound_sockets | 127.22.45.69 | experimental | 127.22.45.69:37677
local_ip_for_outbound_sockets | 127.22.45.70 | experimental | 127.22.45.70:36617
local_ip_for_outbound_sockets | 127.22.45.71 | experimental | 127.22.45.71:39125
never_fsync | true | unsafe,advanced | all 7 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 7 server(s) checked
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 7 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 7 server(s) checked
server_dump_info_format | pb | hidden | all 7 server(s) checked
server_dump_info_path | /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.22.45.65:36905
server_dump_info_path | /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.22.45.66:46041
server_dump_info_path | /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.22.45.67:44809
server_dump_info_path | /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.22.45.68:39501
server_dump_info_path | /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.22.45.69:37677
server_dump_info_path | /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.22.45.70:36617
server_dump_info_path | /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden | 127.22.45.71:39125
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.22.45.84:37045 | all 7 server(s) checked
time_source | builtin | all 7 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 8 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
rebalance_test_table_0 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_4 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 2
First Quartile | 2
Median | 2
Third Quartile | 3
Maximum | 3
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 7
Tables | 6
Tablets | 16
Replicas | 16
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240502 20:45:49.672081 22709 log_verifier.cc:126] Checking tablet 0b736003ce014dbe8e11235e0d8e0872
I20240502 20:45:49.682590 22709 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 0b736003ce014dbe8e11235e0d8e0872
I20240502 20:45:49.682663 22709 log_verifier.cc:126] Checking tablet 21d2ddfb4dea4d32b3fac36a95e6a2b9
I20240502 20:45:49.690577 22709 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 21d2ddfb4dea4d32b3fac36a95e6a2b9
I20240502 20:45:49.690625 22709 log_verifier.cc:126] Checking tablet 3b6e517be2fe4c79bce310d8786554f4
I20240502 20:45:49.698858 22709 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 3b6e517be2fe4c79bce310d8786554f4
I20240502 20:45:49.698910 22709 log_verifier.cc:126] Checking tablet 4a5012394f8e4ac19c07a1093b2dca14
I20240502 20:45:49.706924 22709 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 4a5012394f8e4ac19c07a1093b2dca14
I20240502 20:45:49.706976 22709 log_verifier.cc:126] Checking tablet 61dc146b84774046980dc8fdbf8e632f
I20240502 20:45:49.715109 22709 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 61dc146b84774046980dc8fdbf8e632f
I20240502 20:45:49.715160 22709 log_verifier.cc:126] Checking tablet 71e60972a8354e23adb03542b9935769
I20240502 20:45:49.723346 22709 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 71e60972a8354e23adb03542b9935769
I20240502 20:45:49.723402 22709 log_verifier.cc:126] Checking tablet 77f37012797e41cb9aafd5ab00d049ab
I20240502 20:45:49.731477 22709 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 77f37012797e41cb9aafd5ab00d049ab
I20240502 20:45:49.731528 22709 log_verifier.cc:126] Checking tablet 7ed23be7cabc44b4ad4db0ee4f2e5392
I20240502 20:45:49.739686 22709 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 7ed23be7cabc44b4ad4db0ee4f2e5392
I20240502 20:45:49.739737 22709 log_verifier.cc:126] Checking tablet a95565010b00430d977602e0a485a1dd
I20240502 20:45:49.747761 22709 log_verifier.cc:177] Verified matching terms for 3 ops in tablet a95565010b00430d977602e0a485a1dd
I20240502 20:45:49.747810 22709 log_verifier.cc:126] Checking tablet ca36c671f31b468fb2f68c0f70f4e928
I20240502 20:45:49.756065 22709 log_verifier.cc:177] Verified matching terms for 3 ops in tablet ca36c671f31b468fb2f68c0f70f4e928
I20240502 20:45:49.756116 22709 log_verifier.cc:126] Checking tablet d661eaefcd884d1b947002588ec335ac
I20240502 20:45:49.764339 22709 log_verifier.cc:177] Verified matching terms for 6 ops in tablet d661eaefcd884d1b947002588ec335ac
I20240502 20:45:49.764396 22709 log_verifier.cc:126] Checking tablet dec4ab92da444ee28d3c9ccaa498cb5e
I20240502 20:45:49.771503 22709 log_verifier.cc:177] Verified matching terms for 2 ops in tablet dec4ab92da444ee28d3c9ccaa498cb5e
I20240502 20:45:49.771553 22709 log_verifier.cc:126] Checking tablet e106fd5f2bc44c9e96bc38e2fd0dec1f
I20240502 20:45:49.779924 22709 log_verifier.cc:177] Verified matching terms for 6 ops in tablet e106fd5f2bc44c9e96bc38e2fd0dec1f
I20240502 20:45:49.779976 22709 log_verifier.cc:126] Checking tablet e35840461c9f4b4da8921cb04d283a46
I20240502 20:45:49.788121 22709 log_verifier.cc:177] Verified matching terms for 3 ops in tablet e35840461c9f4b4da8921cb04d283a46
I20240502 20:45:49.788172 22709 log_verifier.cc:126] Checking tablet ee9e40ee38ca422a8ca040a4a49b2ea7
I20240502 20:45:49.796420 22709 log_verifier.cc:177] Verified matching terms for 6 ops in tablet ee9e40ee38ca422a8ca040a4a49b2ea7
I20240502 20:45:49.796470 22709 log_verifier.cc:126] Checking tablet f258bd1f1bd14f99bed69e112cb09358
I20240502 20:45:49.804602 22709 log_verifier.cc:177] Verified matching terms for 6 ops in tablet f258bd1f1bd14f99bed69e112cb09358
I20240502 20:45:49.806538 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.72:0
--local_ip_for_outbound_sockets=127.22.45.72
--webserver_interface=127.22.45.72
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34337
--builtin_ntp_servers=127.22.45.84:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 20:45:49.909217 28245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:49.909411 28245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:49.909451 28245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:49.912401 28245 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:45:49.912458 28245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:49.912539 28245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.72
I20240502 20:45:49.916039 28245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.72:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.22.45.72
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34337
--never_fsync=true
--heap_profile_path=/tmp/kudu.28245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.72
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:49.916821 28245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:49.917920 28245 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:49.926026 28250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:49.926436 28251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:49.926882 28245 server_base.cc:1034] running on GCE node
W20240502 20:45:49.927171 28253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:49.927544 28245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:49.928063 28245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:49.929215 28245 hybrid_clock.cc:648] HybridClock initialized: now 1714682749929202 us; error 28 us; skew 500 ppm
I20240502 20:45:49.930941 28245 webserver.cc:460] Webserver started at http://127.22.45.72:41209/ using document root <none> and password file <none>
I20240502 20:45:49.931453 28245 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:49.931510 28245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:49.931675 28245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:49.933324 28245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "56dc509f2cb74d509732713db87c6a9d"
format_stamp: "Formatted at 2024-05-02 20:45:49 on dist-test-slave-g7g5"
I20240502 20:45:49.933768 28245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "56dc509f2cb74d509732713db87c6a9d"
format_stamp: "Formatted at 2024-05-02 20:45:49 on dist-test-slave-g7g5"
I20240502 20:45:49.936996 28245 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20240502 20:45:49.939050 28259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:49.940038 28245 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:45:49.940140 28245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "56dc509f2cb74d509732713db87c6a9d"
format_stamp: "Formatted at 2024-05-02 20:45:49 on dist-test-slave-g7g5"
I20240502 20:45:49.940223 28245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:49.961632 28245 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:49.962265 28245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:49.962426 28245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:49.963003 28245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:49.963965 28245 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:49.964012 28245 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:49.964053 28245 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:49.964082 28245 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:49.972276 28245 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.72:37665
I20240502 20:45:49.972313 28372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.72:37665 every 8 connection(s)
I20240502 20:45:49.973186 28245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 20:45:49.981657 28373 heartbeater.cc:347] Connected to a master server at 127.22.45.126:34337
I20240502 20:45:49.981781 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 28245
I20240502 20:45:49.981863 28373 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:49.981895 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240502 20:45:49.982316 28373 heartbeater.cc:511] Master 127.22.45.126:34337 requested a full tablet report, sending...
I20240502 20:45:49.983290 24572 ts_manager.cc:194] Registered new tserver with Master: 56dc509f2cb74d509732713db87c6a9d (127.22.45.72:37665)
I20240502 20:45:49.983871 24572 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.72:56989
I20240502 20:45:50.165386 24696 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:50.168458 25834 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:50.171413 26369 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:50.172930 26237 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:50.182435 28307 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:50.183987 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:50.198817 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:50.208976 24828 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:50.307933 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:50.308707 28307 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:50.310092 26369 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:50.315560 24696 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:50.317293 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:50.323001 25834 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:50.343381 26237 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:50.343957 24828 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:50.381108 24848 consensus_queue.cc:237] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "56dc509f2cb74d509732713db87c6a9d" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 37665 } }
W20240502 20:45:50.385289 24781 consensus_peers.cc:485] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4 -> Peer 56dc509f2cb74d509732713db87c6a9d (127.22.45.72:37665): Couldn't send request to peer 56dc509f2cb74d509732713db87c6a9d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ca36c671f31b468fb2f68c0f70f4e928. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:45:50.963213 28479 ts_tablet_manager.cc:902] T ca36c671f31b468fb2f68c0f70f4e928 P 56dc509f2cb74d509732713db87c6a9d: Initiating tablet copy from peer f926d9d9d2314c2b90437934356f66d4 (127.22.45.66:46041)
I20240502 20:45:50.963872 28479 tablet_copy_client.cc:308] T ca36c671f31b468fb2f68c0f70f4e928 P 56dc509f2cb74d509732713db87c6a9d: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.66:46041
I20240502 20:45:50.967397 24868 tablet_copy_service.cc:140] P f926d9d9d2314c2b90437934356f66d4: Received BeginTabletCopySession request for tablet ca36c671f31b468fb2f68c0f70f4e928 from peer 56dc509f2cb74d509732713db87c6a9d ({username='slave'} at 127.22.45.72:33015)
I20240502 20:45:50.967495 24868 tablet_copy_service.cc:161] P f926d9d9d2314c2b90437934356f66d4: Beginning new tablet copy session on tablet ca36c671f31b468fb2f68c0f70f4e928 from peer 56dc509f2cb74d509732713db87c6a9d at {username='slave'} at 127.22.45.72:33015: session id = 56dc509f2cb74d509732713db87c6a9d-ca36c671f31b468fb2f68c0f70f4e928
I20240502 20:45:50.968168 24868 tablet_copy_source_session.cc:205] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:45:50.971071 28479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca36c671f31b468fb2f68c0f70f4e928. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:50.976406 28479 tablet_copy_client.cc:783] T ca36c671f31b468fb2f68c0f70f4e928 P 56dc509f2cb74d509732713db87c6a9d: tablet copy: Starting download of 0 data blocks...
I20240502 20:45:50.976610 28479 tablet_copy_client.cc:647] T ca36c671f31b468fb2f68c0f70f4e928 P 56dc509f2cb74d509732713db87c6a9d: tablet copy: Starting download of 1 WAL segments...
I20240502 20:45:50.978027 28479 tablet_copy_client.cc:523] T ca36c671f31b468fb2f68c0f70f4e928 P 56dc509f2cb74d509732713db87c6a9d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:45:50.979823 28479 tablet_bootstrap.cc:492] T ca36c671f31b468fb2f68c0f70f4e928 P 56dc509f2cb74d509732713db87c6a9d: Bootstrap starting.
I20240502 20:45:50.985200 28479 log.cc:824] T ca36c671f31b468fb2f68c0f70f4e928 P 56dc509f2cb74d509732713db87c6a9d: Log is configured to *not* fsync() on all Append() calls
I20240502 20:45:50.985994 28373 heartbeater.cc:503] Master 127.22.45.126:34337 was elected leader, sending a full tablet report...
I20240502 20:45:51.003495 28479 tablet_bootstrap.cc:492] T ca36c671f31b468fb2f68c0f70f4e928 P 56dc509f2cb74d509732713db87c6a9d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 20:45:51.003813 28479 tablet_bootstrap.cc:492] T ca36c671f31b468fb2f68c0f70f4e928 P 56dc509f2cb74d509732713db87c6a9d: Bootstrap complete.
I20240502 20:45:51.004062 28479 ts_tablet_manager.cc:1367] T ca36c671f31b468fb2f68c0f70f4e928 P 56dc509f2cb74d509732713db87c6a9d: Time spent bootstrapping tablet: real 0.024s user 0.020s sys 0.004s
I20240502 20:45:51.005851 28479 raft_consensus.cc:348] T ca36c671f31b468fb2f68c0f70f4e928 P 56dc509f2cb74d509732713db87c6a9d [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } }
I20240502 20:45:51.006374 28479 raft_consensus.cc:729] T ca36c671f31b468fb2f68c0f70f4e928 P 56dc509f2cb74d509732713db87c6a9d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56dc509f2cb74d509732713db87c6a9d, State: Initialized, Role: FOLLOWER
I20240502 20:45:51.006668 28479 consensus_queue.cc:260] T ca36c671f31b468fb2f68c0f70f4e928 P 56dc509f2cb74d509732713db87c6a9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "56dc509f2cb74d509732713db87c6a9d" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 37665 } }
I20240502 20:45:51.007441 28479 ts_tablet_manager.cc:1398] T ca36c671f31b468fb2f68c0f70f4e928 P 56dc509f2cb74d509732713db87c6a9d: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20240502 20:45:51.008015 24868 tablet_copy_service.cc:342] P f926d9d9d2314c2b90437934356f66d4: Request end of tablet copy session 56dc509f2cb74d509732713db87c6a9d-ca36c671f31b468fb2f68c0f70f4e928 received from {username='slave'} at 127.22.45.72:33015
I20240502 20:45:51.008090 24868 tablet_copy_service.cc:433] P f926d9d9d2314c2b90437934356f66d4: ending tablet copy session 56dc509f2cb74d509732713db87c6a9d-ca36c671f31b468fb2f68c0f70f4e928 on tablet ca36c671f31b468fb2f68c0f70f4e928 with peer 56dc509f2cb74d509732713db87c6a9d
I20240502 20:45:51.359231 28327 raft_consensus.cc:1203] T ca36c671f31b468fb2f68c0f70f4e928 P 56dc509f2cb74d509732713db87c6a9d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:45:51.360411 28488 raft_consensus.cc:2932] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 56dc509f2cb74d509732713db87c6a9d (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "56dc509f2cb74d509732713db87c6a9d" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 37665 } } }
I20240502 20:45:51.361032 28327 raft_consensus.cc:2932] T ca36c671f31b468fb2f68c0f70f4e928 P 56dc509f2cb74d509732713db87c6a9d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 56dc509f2cb74d509732713db87c6a9d (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "56dc509f2cb74d509732713db87c6a9d" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 37665 } } }
I20240502 20:45:51.363423 24567 catalog_manager.cc:5484] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4 reported cstate change: config changed from index -1 to 4, VOTER 56dc509f2cb74d509732713db87c6a9d (127.22.45.72) added. New cstate: current_term: 1 leader_uuid: "f926d9d9d2314c2b90437934356f66d4" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "56dc509f2cb74d509732713db87c6a9d" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 37665 } } }
I20240502 20:45:51.384884 24716 consensus_queue.cc:237] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "56dc509f2cb74d509732713db87c6a9d" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 37665 } }
W20240502 20:45:51.389050 24649 consensus_peers.cc:485] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77 -> Peer 56dc509f2cb74d509732713db87c6a9d (127.22.45.72:37665): Couldn't send request to peer 56dc509f2cb74d509732713db87c6a9d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a95565010b00430d977602e0a485a1dd. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:45:51.820106 28515 ts_tablet_manager.cc:902] T a95565010b00430d977602e0a485a1dd P 56dc509f2cb74d509732713db87c6a9d: Initiating tablet copy from peer 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65:36905)
I20240502 20:45:51.820580 28515 tablet_copy_client.cc:308] T a95565010b00430d977602e0a485a1dd P 56dc509f2cb74d509732713db87c6a9d: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.65:36905
I20240502 20:45:51.824121 24736 tablet_copy_service.cc:140] P 1c52358e8e2f4ffe992cafe0f1a95a77: Received BeginTabletCopySession request for tablet a95565010b00430d977602e0a485a1dd from peer 56dc509f2cb74d509732713db87c6a9d ({username='slave'} at 127.22.45.72:54667)
I20240502 20:45:51.824218 24736 tablet_copy_service.cc:161] P 1c52358e8e2f4ffe992cafe0f1a95a77: Beginning new tablet copy session on tablet a95565010b00430d977602e0a485a1dd from peer 56dc509f2cb74d509732713db87c6a9d at {username='slave'} at 127.22.45.72:54667: session id = 56dc509f2cb74d509732713db87c6a9d-a95565010b00430d977602e0a485a1dd
I20240502 20:45:51.824942 24736 tablet_copy_source_session.cc:205] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:45:51.825732 28515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a95565010b00430d977602e0a485a1dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:51.827386 28515 tablet_copy_client.cc:783] T a95565010b00430d977602e0a485a1dd P 56dc509f2cb74d509732713db87c6a9d: tablet copy: Starting download of 0 data blocks...
I20240502 20:45:51.827523 28515 tablet_copy_client.cc:647] T a95565010b00430d977602e0a485a1dd P 56dc509f2cb74d509732713db87c6a9d: tablet copy: Starting download of 1 WAL segments...
I20240502 20:45:51.828547 28515 tablet_copy_client.cc:523] T a95565010b00430d977602e0a485a1dd P 56dc509f2cb74d509732713db87c6a9d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:45:51.829442 28515 tablet_bootstrap.cc:492] T a95565010b00430d977602e0a485a1dd P 56dc509f2cb74d509732713db87c6a9d: Bootstrap starting.
I20240502 20:45:51.846613 28515 tablet_bootstrap.cc:492] T a95565010b00430d977602e0a485a1dd P 56dc509f2cb74d509732713db87c6a9d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 20:45:51.846812 28515 tablet_bootstrap.cc:492] T a95565010b00430d977602e0a485a1dd P 56dc509f2cb74d509732713db87c6a9d: Bootstrap complete.
I20240502 20:45:51.846931 28515 ts_tablet_manager.cc:1367] T a95565010b00430d977602e0a485a1dd P 56dc509f2cb74d509732713db87c6a9d: Time spent bootstrapping tablet: real 0.018s user 0.018s sys 0.000s
I20240502 20:45:51.847349 28515 raft_consensus.cc:348] T a95565010b00430d977602e0a485a1dd P 56dc509f2cb74d509732713db87c6a9d [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } }
I20240502 20:45:51.847529 28515 raft_consensus.cc:729] T a95565010b00430d977602e0a485a1dd P 56dc509f2cb74d509732713db87c6a9d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56dc509f2cb74d509732713db87c6a9d, State: Initialized, Role: FOLLOWER
I20240502 20:45:51.847605 28515 consensus_queue.cc:260] T a95565010b00430d977602e0a485a1dd P 56dc509f2cb74d509732713db87c6a9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "56dc509f2cb74d509732713db87c6a9d" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 37665 } }
I20240502 20:45:51.847824 28515 ts_tablet_manager.cc:1398] T a95565010b00430d977602e0a485a1dd P 56dc509f2cb74d509732713db87c6a9d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:45:51.848307 24736 tablet_copy_service.cc:342] P 1c52358e8e2f4ffe992cafe0f1a95a77: Request end of tablet copy session 56dc509f2cb74d509732713db87c6a9d-a95565010b00430d977602e0a485a1dd received from {username='slave'} at 127.22.45.72:54667
I20240502 20:45:51.848377 24736 tablet_copy_service.cc:433] P 1c52358e8e2f4ffe992cafe0f1a95a77: ending tablet copy session 56dc509f2cb74d509732713db87c6a9d-a95565010b00430d977602e0a485a1dd on tablet a95565010b00430d977602e0a485a1dd with peer 56dc509f2cb74d509732713db87c6a9d
I20240502 20:45:52.411003 28327 raft_consensus.cc:1203] T a95565010b00430d977602e0a485a1dd P 56dc509f2cb74d509732713db87c6a9d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:45:52.411854 28525 raft_consensus.cc:2932] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 56dc509f2cb74d509732713db87c6a9d (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "56dc509f2cb74d509732713db87c6a9d" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 37665 } } }
I20240502 20:45:52.412354 28327 raft_consensus.cc:2932] T a95565010b00430d977602e0a485a1dd P 56dc509f2cb74d509732713db87c6a9d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 56dc509f2cb74d509732713db87c6a9d (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "56dc509f2cb74d509732713db87c6a9d" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 37665 } } }
I20240502 20:45:52.413234 28525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:59488 (request call id 0) took 1028 ms. Trace:
I20240502 20:45:52.413345 28525 rpcz_store.cc:276] 0502 20:45:51.384580 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 20:45:51.384634 (+ 54us) service_pool.cc:225] Handling call
0502 20:45:51.385316 (+ 682us) log.cc:842] Serialized 257 byte log entry
0502 20:45:52.413216 (+1027900us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":43,"raft.queue_time_us":96,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":116,"thread_start_us":100,"threads_started":2,"wal-append.queue_time_us":131,"wal-append.run_cpu_time_us":281,"wal-append.run_wall_time_us":1000340}
I20240502 20:45:52.414673 24572 catalog_manager.cc:5484] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77 reported cstate change: config changed from index -1 to 4, VOTER 56dc509f2cb74d509732713db87c6a9d (127.22.45.72) added. New cstate: current_term: 1 leader_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "56dc509f2cb74d509732713db87c6a9d" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 37665 } } }
I20240502 20:45:52.468524 24696 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:52.469890 28307 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:52.470206 25834 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:52.470273 26369 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:52.471462 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:52.483656 24828 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:52.491469 26237 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:52.492858 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:52.523998 24716 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a95565010b00430d977602e0a485a1dd"
dest_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:59516
I20240502 20:45:52.524133 24716 raft_consensus.cc:595] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Received request to transfer leadership
I20240502 20:45:52.572299 26369 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:52.578997 28307 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:52.579890 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:52.582123 24696 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:52.582908 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:52.583844 24828 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:52.590652 26237 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:52.599491 25834 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:52.686492 25834 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:52.688122 24696 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:52.688897 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:52.694658 26369 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:52.696215 28307 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:52.698280 26237 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:52.698865 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:52.719151 24828 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:52.739495 24848 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ca36c671f31b468fb2f68c0f70f4e928"
dest_uuid: "f926d9d9d2314c2b90437934356f66d4"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:59224
I20240502 20:45:52.739624 24848 raft_consensus.cc:595] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Received request to transfer leadership
I20240502 20:45:52.788707 24696 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:52.790663 24828 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:45:52.791514 26369 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:52.804926 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:52.805953 26237 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:52.806531 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:52.807546 25834 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:52.819443 28307 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:45:52.913986 28525 raft_consensus.cc:979] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77: : Instructing follower 56dc509f2cb74d509732713db87c6a9d to start an election
I20240502 20:45:52.914077 28525 raft_consensus.cc:1067] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Signalling peer 56dc509f2cb74d509732713db87c6a9d to start an election
I20240502 20:45:52.914558 28327 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a95565010b00430d977602e0a485a1dd"
dest_uuid: "56dc509f2cb74d509732713db87c6a9d"
from {username='slave'} at 127.22.45.65:53253
I20240502 20:45:52.914738 28327 raft_consensus.cc:482] T a95565010b00430d977602e0a485a1dd P 56dc509f2cb74d509732713db87c6a9d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:45:52.914822 28327 raft_consensus.cc:3037] T a95565010b00430d977602e0a485a1dd P 56dc509f2cb74d509732713db87c6a9d [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:45:52.915608 28327 raft_consensus.cc:504] T a95565010b00430d977602e0a485a1dd P 56dc509f2cb74d509732713db87c6a9d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "56dc509f2cb74d509732713db87c6a9d" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 37665 } }
I20240502 20:45:52.916491 28327 leader_election.cc:288] T a95565010b00430d977602e0a485a1dd P 56dc509f2cb74d509732713db87c6a9d [CANDIDATE]: Term 2 election: Requested vote from peers 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65:36905)
I20240502 20:45:52.916878 24716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a95565010b00430d977602e0a485a1dd" candidate_uuid: "56dc509f2cb74d509732713db87c6a9d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77"
I20240502 20:45:52.916994 24716 raft_consensus.cc:3032] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:45:52.917032 24716 raft_consensus.cc:729] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1c52358e8e2f4ffe992cafe0f1a95a77, State: Running, Role: LEADER
I20240502 20:45:52.917165 24716 consensus_queue.cc:260] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "56dc509f2cb74d509732713db87c6a9d" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 37665 } }
I20240502 20:45:52.917289 24716 raft_consensus.cc:3037] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:45:52.918007 24716 raft_consensus.cc:2454] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56dc509f2cb74d509732713db87c6a9d in term 2.
I20240502 20:45:52.918349 28263 leader_election.cc:302] T a95565010b00430d977602e0a485a1dd P 56dc509f2cb74d509732713db87c6a9d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1c52358e8e2f4ffe992cafe0f1a95a77, 56dc509f2cb74d509732713db87c6a9d; no voters:
I20240502 20:45:52.918699 28695 raft_consensus.cc:2781] T a95565010b00430d977602e0a485a1dd P 56dc509f2cb74d509732713db87c6a9d [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:45:52.918870 28695 raft_consensus.cc:686] T a95565010b00430d977602e0a485a1dd P 56dc509f2cb74d509732713db87c6a9d [term 2 LEADER]: Becoming Leader. State: Replica: 56dc509f2cb74d509732713db87c6a9d, State: Running, Role: LEADER
I20240502 20:45:52.919132 28695 consensus_queue.cc:237] T a95565010b00430d977602e0a485a1dd P 56dc509f2cb74d509732713db87c6a9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "56dc509f2cb74d509732713db87c6a9d" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 37665 } }
I20240502 20:45:52.920877 24568 catalog_manager.cc:5484] T a95565010b00430d977602e0a485a1dd P 56dc509f2cb74d509732713db87c6a9d reported cstate change: term changed from 1 to 2, leader changed from 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65) to 56dc509f2cb74d509732713db87c6a9d (127.22.45.72). New cstate: current_term: 2 leader_uuid: "56dc509f2cb74d509732713db87c6a9d" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 } } peers { permanent_uuid: "56dc509f2cb74d509732713db87c6a9d" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 37665 } } }
I20240502 20:45:52.925666 28700 raft_consensus.cc:979] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4: : Instructing follower 56dc509f2cb74d509732713db87c6a9d to start an election
I20240502 20:45:52.925750 28700 raft_consensus.cc:1067] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Signalling peer 56dc509f2cb74d509732713db87c6a9d to start an election
I20240502 20:45:52.926136 28327 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ca36c671f31b468fb2f68c0f70f4e928"
dest_uuid: "56dc509f2cb74d509732713db87c6a9d"
from {username='slave'} at 127.22.45.66:51869
I20240502 20:45:52.926242 28327 raft_consensus.cc:482] T ca36c671f31b468fb2f68c0f70f4e928 P 56dc509f2cb74d509732713db87c6a9d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:45:52.926295 28327 raft_consensus.cc:3037] T ca36c671f31b468fb2f68c0f70f4e928 P 56dc509f2cb74d509732713db87c6a9d [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:45:52.926995 28327 raft_consensus.cc:504] T ca36c671f31b468fb2f68c0f70f4e928 P 56dc509f2cb74d509732713db87c6a9d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "56dc509f2cb74d509732713db87c6a9d" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 37665 } }
I20240502 20:45:52.927282 28327 leader_election.cc:288] T ca36c671f31b468fb2f68c0f70f4e928 P 56dc509f2cb74d509732713db87c6a9d [CANDIDATE]: Term 2 election: Requested vote from peers f926d9d9d2314c2b90437934356f66d4 (127.22.45.66:46041)
I20240502 20:45:52.927630 24848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca36c671f31b468fb2f68c0f70f4e928" candidate_uuid: "56dc509f2cb74d509732713db87c6a9d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f926d9d9d2314c2b90437934356f66d4"
I20240502 20:45:52.927732 24848 raft_consensus.cc:3032] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:45:52.927764 24848 raft_consensus.cc:729] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f926d9d9d2314c2b90437934356f66d4, State: Running, Role: LEADER
I20240502 20:45:52.927870 24848 consensus_queue.cc:260] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "56dc509f2cb74d509732713db87c6a9d" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 37665 } }
I20240502 20:45:52.927968 24848 raft_consensus.cc:3037] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:45:52.928622 24848 raft_consensus.cc:2454] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56dc509f2cb74d509732713db87c6a9d in term 2.
I20240502 20:45:52.928926 28261 leader_election.cc:302] T ca36c671f31b468fb2f68c0f70f4e928 P 56dc509f2cb74d509732713db87c6a9d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 56dc509f2cb74d509732713db87c6a9d, f926d9d9d2314c2b90437934356f66d4; no voters:
I20240502 20:45:52.929096 28695 raft_consensus.cc:2781] T ca36c671f31b468fb2f68c0f70f4e928 P 56dc509f2cb74d509732713db87c6a9d [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:45:52.929180 28695 raft_consensus.cc:686] T ca36c671f31b468fb2f68c0f70f4e928 P 56dc509f2cb74d509732713db87c6a9d [term 2 LEADER]: Becoming Leader. State: Replica: 56dc509f2cb74d509732713db87c6a9d, State: Running, Role: LEADER
I20240502 20:45:52.929301 28695 consensus_queue.cc:237] T ca36c671f31b468fb2f68c0f70f4e928 P 56dc509f2cb74d509732713db87c6a9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "56dc509f2cb74d509732713db87c6a9d" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 37665 } }
I20240502 20:45:52.930548 24568 catalog_manager.cc:5484] T ca36c671f31b468fb2f68c0f70f4e928 P 56dc509f2cb74d509732713db87c6a9d reported cstate change: term changed from 1 to 2, leader changed from f926d9d9d2314c2b90437934356f66d4 (127.22.45.66) to 56dc509f2cb74d509732713db87c6a9d (127.22.45.72). New cstate: current_term: 2 leader_uuid: "56dc509f2cb74d509732713db87c6a9d" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 } } peers { permanent_uuid: "56dc509f2cb74d509732713db87c6a9d" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 37665 } } }
I20240502 20:45:53.099556 24696 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.100108 26369 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.105902 28307 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.106415 25834 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.111719 26237 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.112444 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.115777 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.128392 24828 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.206395 24696 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.208431 28307 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.212486 26369 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.216405 24828 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.220443 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.228322 25834 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.235657 26237 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.236634 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.373701 24716 raft_consensus.cc:1261] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 2 FOLLOWER]: Refusing update from remote peer 56dc509f2cb74d509732713db87c6a9d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:45:53.374246 28695 consensus_queue.cc:1035] T a95565010b00430d977602e0a485a1dd P 56dc509f2cb74d509732713db87c6a9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c52358e8e2f4ffe992cafe0f1a95a77" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 36905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:45:53.497287 24848 raft_consensus.cc:1261] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4 [term 2 FOLLOWER]: Refusing update from remote peer 56dc509f2cb74d509732713db87c6a9d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:45:53.497746 28696 consensus_queue.cc:1035] T ca36c671f31b468fb2f68c0f70f4e928 P 56dc509f2cb74d509732713db87c6a9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f926d9d9d2314c2b90437934356f66d4" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:45:53.520217 28307 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.520341 24696 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.523361 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.526729 26369 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.530046 26237 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.539326 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.540387 24828 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.560760 25834 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.592005 28327 consensus_queue.cc:237] T a95565010b00430d977602e0a485a1dd P 56dc509f2cb74d509732713db87c6a9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "56dc509f2cb74d509732713db87c6a9d" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 37665 } }
I20240502 20:45:53.592604 28695 raft_consensus.cc:2932] T a95565010b00430d977602e0a485a1dd P 56dc509f2cb74d509732713db87c6a9d [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "56dc509f2cb74d509732713db87c6a9d" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 37665 } } }
I20240502 20:45:53.594801 24564 catalog_manager.cc:5484] T a95565010b00430d977602e0a485a1dd P 56dc509f2cb74d509732713db87c6a9d reported cstate change: config changed from index 4 to 6, VOTER 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65) evicted. New cstate: current_term: 2 leader_uuid: "56dc509f2cb74d509732713db87c6a9d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "56dc509f2cb74d509732713db87c6a9d" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 37665 } } }
I20240502 20:45:53.597410 24696 tablet_service.cc:1514] Processing DeleteTablet for tablet a95565010b00430d977602e0a485a1dd with delete_type TABLET_DATA_TOMBSTONED (TS 1c52358e8e2f4ffe992cafe0f1a95a77 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:34622
I20240502 20:45:53.597777 28845 tablet_replica.cc:330] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77: stopping tablet replica
I20240502 20:45:53.597911 28845 raft_consensus.cc:2229] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 20:45:53.598030 28845 raft_consensus.cc:2258] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 20:45:53.599500 28845 ts_tablet_manager.cc:1875] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:45:53.601375 28845 ts_tablet_manager.cc:1888] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 20:45:53.601609 28845 log.cc:1196] T a95565010b00430d977602e0a485a1dd P 1c52358e8e2f4ffe992cafe0f1a95a77: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/a95565010b00430d977602e0a485a1dd
I20240502 20:45:53.602124 24560 catalog_manager.cc:4830] TS 1c52358e8e2f4ffe992cafe0f1a95a77 (127.22.45.65:36905): tablet a95565010b00430d977602e0a485a1dd (table rebalance_test_table_1 [id=2214dd294ae048a4926821c50ed5e793]) successfully deleted
I20240502 20:45:53.648579 24696 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.649235 25834 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.649456 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.651969 28307 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.661775 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.665983 26237 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.667805 26369 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.680215 24828 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.713002 28327 consensus_queue.cc:237] T ca36c671f31b468fb2f68c0f70f4e928 P 56dc509f2cb74d509732713db87c6a9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "56dc509f2cb74d509732713db87c6a9d" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 37665 } }
I20240502 20:45:53.713565 28695 raft_consensus.cc:2932] T ca36c671f31b468fb2f68c0f70f4e928 P 56dc509f2cb74d509732713db87c6a9d [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER f926d9d9d2314c2b90437934356f66d4 (127.22.45.66) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "56dc509f2cb74d509732713db87c6a9d" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 37665 } } }
I20240502 20:45:53.715562 24568 catalog_manager.cc:5484] T ca36c671f31b468fb2f68c0f70f4e928 P 56dc509f2cb74d509732713db87c6a9d reported cstate change: config changed from index 4 to 6, VOTER f926d9d9d2314c2b90437934356f66d4 (127.22.45.66) evicted. New cstate: current_term: 2 leader_uuid: "56dc509f2cb74d509732713db87c6a9d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "56dc509f2cb74d509732713db87c6a9d" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 37665 } } }
I20240502 20:45:53.717752 24828 tablet_service.cc:1514] Processing DeleteTablet for tablet ca36c671f31b468fb2f68c0f70f4e928 with delete_type TABLET_DATA_TOMBSTONED (TS f926d9d9d2314c2b90437934356f66d4 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:35518
I20240502 20:45:53.718111 28894 tablet_replica.cc:330] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4: stopping tablet replica
I20240502 20:45:53.718216 28894 raft_consensus.cc:2229] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 20:45:53.718305 28894 raft_consensus.cc:2258] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 20:45:53.719457 28894 ts_tablet_manager.cc:1875] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:45:53.721158 28894 ts_tablet_manager.cc:1888] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 20:45:53.721223 28894 log.cc:1196] T ca36c671f31b468fb2f68c0f70f4e928 P f926d9d9d2314c2b90437934356f66d4: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/ca36c671f31b468fb2f68c0f70f4e928
I20240502 20:45:53.721654 24558 catalog_manager.cc:4830] TS f926d9d9d2314c2b90437934356f66d4 (127.22.45.66:46041): tablet ca36c671f31b468fb2f68c0f70f4e928 (table rebalance_test_table_4 [id=493ba88faedf41dcb0c39446bb42569d]) successfully deleted
I20240502 20:45:53.977969 25834 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.979779 28307 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.982695 26369 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.988332 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.989311 24696 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:53.998732 26237 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:54.005843 24828 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:54.010605 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:54.095569 24696 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:54.095610 28307 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:54.096701 26369 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:54.102759 25834 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:54.113389 24828 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:54.113647 26237 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:54.113802 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:54.138378 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:54.216145 28307 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:54.216737 24696 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:54.218091 26369 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:54.219405 25834 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:54.226966 24828 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:54.236444 26237 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:54.237763 25973 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:45:54.253715 26105 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
fe9820ffe81d4edb865d80b0d7faae0c | 127.22.45.126:34337 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
allow_unsafe_replication_factor | true | runtime,unsafe | all 1 server(s) checked
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.22.45.84:37045 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
1c52358e8e2f4ffe992cafe0f1a95a77 | 127.22.45.65:36905 | HEALTHY | <none> | 2 | 0
56dc509f2cb74d509732713db87c6a9d | 127.22.45.72:37665 | HEALTHY | <none> | 2 | 0
85b4f683f6904c35b3c0e91799295bb3 | 127.22.45.71:39125 | HEALTHY | <none> | 2 | 0
a984b4b59cb6429290bc907a860d50f8 | 127.22.45.67:44809 | HEALTHY | <none> | 2 | 0
bb697f05a8394524a411ece19fe608dc | 127.22.45.68:39501 | HEALTHY | <none> | 2 | 0
eef5daac82024c449fc8b0dc82bd428c | 127.22.45.69:37677 | HEALTHY | <none> | 2 | 0
f6dd93d055454bc5965b407745c020ff | 127.22.45.70:36617 | HEALTHY | <none> | 2 | 0
f926d9d9d2314c2b90437934356f66d4 | 127.22.45.66:46041 | HEALTHY | <none> | 2 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 8
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 8 server(s) checked
local_ip_for_outbound_sockets | 127.22.45.65 | experimental | 127.22.45.65:36905
local_ip_for_outbound_sockets | 127.22.45.66 | experimental | 127.22.45.66:46041
local_ip_for_outbound_sockets | 127.22.45.67 | experimental | 127.22.45.67:44809
local_ip_for_outbound_sockets | 127.22.45.68 | experimental | 127.22.45.68:39501
local_ip_for_outbound_sockets | 127.22.45.69 | experimental | 127.22.45.69:37677
local_ip_for_outbound_sockets | 127.22.45.70 | experimental | 127.22.45.70:36617
local_ip_for_outbound_sockets | 127.22.45.71 | experimental | 127.22.45.71:39125
local_ip_for_outbound_sockets | 127.22.45.72 | experimental | 127.22.45.72:37665
never_fsync | true | unsafe,advanced | all 8 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 8 server(s) checked
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 8 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 8 server(s) checked
server_dump_info_format | pb | hidden | all 8 server(s) checked
server_dump_info_path | /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.22.45.65:36905
server_dump_info_path | /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.22.45.66:46041
server_dump_info_path | /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.22.45.67:44809
server_dump_info_path | /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.22.45.68:39501
server_dump_info_path | /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.22.45.69:37677
server_dump_info_path | /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.22.45.70:36617
server_dump_info_path | /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden | 127.22.45.71:39125
server_dump_info_path | /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data/info.pb | hidden | 127.22.45.72:37665
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.22.45.84:37045 | all 8 server(s) checked
time_source | builtin | all 8 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 9 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
rebalance_test_table_0 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_4 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 2
First Quartile | 2
Median | 2
Third Quartile | 2
Maximum | 2
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 8
Tables | 6
Tablets | 16
Replicas | 16
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240502 20:45:54.278738 22709 log_verifier.cc:126] Checking tablet 0b736003ce014dbe8e11235e0d8e0872
I20240502 20:45:54.284794 22709 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 0b736003ce014dbe8e11235e0d8e0872
I20240502 20:45:54.284837 22709 log_verifier.cc:126] Checking tablet 21d2ddfb4dea4d32b3fac36a95e6a2b9
I20240502 20:45:54.290263 22709 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 21d2ddfb4dea4d32b3fac36a95e6a2b9
I20240502 20:45:54.290304 22709 log_verifier.cc:126] Checking tablet 3b6e517be2fe4c79bce310d8786554f4
I20240502 20:45:54.295713 22709 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 3b6e517be2fe4c79bce310d8786554f4
I20240502 20:45:54.295758 22709 log_verifier.cc:126] Checking tablet 4a5012394f8e4ac19c07a1093b2dca14
I20240502 20:45:54.300993 22709 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 4a5012394f8e4ac19c07a1093b2dca14
I20240502 20:45:54.301034 22709 log_verifier.cc:126] Checking tablet 61dc146b84774046980dc8fdbf8e632f
I20240502 20:45:54.306372 22709 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 61dc146b84774046980dc8fdbf8e632f
I20240502 20:45:54.306416 22709 log_verifier.cc:126] Checking tablet 71e60972a8354e23adb03542b9935769
I20240502 20:45:54.311820 22709 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 71e60972a8354e23adb03542b9935769
I20240502 20:45:54.311864 22709 log_verifier.cc:126] Checking tablet 77f37012797e41cb9aafd5ab00d049ab
I20240502 20:45:54.317194 22709 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 77f37012797e41cb9aafd5ab00d049ab
I20240502 20:45:54.317235 22709 log_verifier.cc:126] Checking tablet 7ed23be7cabc44b4ad4db0ee4f2e5392
I20240502 20:45:54.322652 22709 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 7ed23be7cabc44b4ad4db0ee4f2e5392
I20240502 20:45:54.322690 22709 log_verifier.cc:126] Checking tablet a95565010b00430d977602e0a485a1dd
I20240502 20:45:54.330343 22709 log_verifier.cc:177] Verified matching terms for 6 ops in tablet a95565010b00430d977602e0a485a1dd
I20240502 20:45:54.330389 22709 log_verifier.cc:126] Checking tablet ca36c671f31b468fb2f68c0f70f4e928
I20240502 20:45:54.338016 22709 log_verifier.cc:177] Verified matching terms for 6 ops in tablet ca36c671f31b468fb2f68c0f70f4e928
I20240502 20:45:54.338066 22709 log_verifier.cc:126] Checking tablet d661eaefcd884d1b947002588ec335ac
I20240502 20:45:54.343531 22709 log_verifier.cc:177] Verified matching terms for 6 ops in tablet d661eaefcd884d1b947002588ec335ac
I20240502 20:45:54.343575 22709 log_verifier.cc:126] Checking tablet dec4ab92da444ee28d3c9ccaa498cb5e
I20240502 20:45:54.347837 22709 log_verifier.cc:177] Verified matching terms for 2 ops in tablet dec4ab92da444ee28d3c9ccaa498cb5e
I20240502 20:45:54.347884 22709 log_verifier.cc:126] Checking tablet e106fd5f2bc44c9e96bc38e2fd0dec1f
I20240502 20:45:54.353754 22709 log_verifier.cc:177] Verified matching terms for 6 ops in tablet e106fd5f2bc44c9e96bc38e2fd0dec1f
I20240502 20:45:54.353806 22709 log_verifier.cc:126] Checking tablet e35840461c9f4b4da8921cb04d283a46
I20240502 20:45:54.359194 22709 log_verifier.cc:177] Verified matching terms for 3 ops in tablet e35840461c9f4b4da8921cb04d283a46
I20240502 20:45:54.359234 22709 log_verifier.cc:126] Checking tablet ee9e40ee38ca422a8ca040a4a49b2ea7
I20240502 20:45:54.364468 22709 log_verifier.cc:177] Verified matching terms for 6 ops in tablet ee9e40ee38ca422a8ca040a4a49b2ea7
I20240502 20:45:54.364509 22709 log_verifier.cc:126] Checking tablet f258bd1f1bd14f99bed69e112cb09358
I20240502 20:45:54.369814 22709 log_verifier.cc:177] Verified matching terms for 6 ops in tablet f258bd1f1bd14f99bed69e112cb09358
I20240502 20:45:54.370236 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 24634
I20240502 20:45:54.376468 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 24766
I20240502 20:45:54.382037 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 25771
I20240502 20:45:54.387081 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 25905
I20240502 20:45:54.391673 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 26043
I20240502 20:45:54.396123 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 26175
I20240502 20:45:54.400384 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 26307
I20240502 20:45:54.404664 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 28245
I20240502 20:45:54.409991 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 24542
2024-05-02T20:45:54Z chronyd exiting
[ OK ] RebalanceParamTest.Rebalance/0 (25594 ms)
[----------] 1 test from RebalanceParamTest (25594 ms total)
[----------] 1 test from DDLDuringRebalancingTest
[ RUN ] DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing/0
I20240502 20:45:54.443619 22709 test_util.cc:268] Using random seed: -1334105763
I20240502 20:45:54.444358 22709 ts_itest-base.cc:112] Starting cluster with:
I20240502 20:45:54.444393 22709 ts_itest-base.cc:113] --------------
I20240502 20:45:54.444403 22709 ts_itest-base.cc:114] 8 tablet servers
I20240502 20:45:54.444411 22709 ts_itest-base.cc:115] 3 replicas per TS
I20240502 20:45:54.444422 22709 ts_itest-base.cc:116] --------------
2024-05-02T20:45:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T20:45:54Z Disabled control of system clock
I20240502 20:45:54.452085 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.45.126:34109
--webserver_interface=127.22.45.126
--webserver_port=0
--builtin_ntp_servers=127.22.45.84:45597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.45.126:34109
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=false
--enable_range_replica_placement=false with env {}
W20240502 20:45:54.552919 29011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:54.553120 29011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:54.553170 29011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:54.556138 29011 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:45:54.556203 29011 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 20:45:54.556247 29011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:54.556286 29011 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 20:45:54.556324 29011 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 20:45:54.559862 29011 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:45597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.22.45.126:34109
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.45.126:34109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.22.45.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.29011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:54.560645 29011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:54.561699 29011 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:54.568583 29019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:54.568933 29016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:54.569342 29017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:54.569770 29011 server_base.cc:1034] running on GCE node
I20240502 20:45:54.570156 29011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:54.570977 29011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:54.572114 29011 hybrid_clock.cc:648] HybridClock initialized: now 1714682754572098 us; error 31 us; skew 500 ppm
I20240502 20:45:54.573694 29011 webserver.cc:460] Webserver started at http://127.22.45.126:39307/ using document root <none> and password file <none>
I20240502 20:45:54.574195 29011 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:54.574246 29011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:54.574417 29011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:54.576013 29011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "67913548da9f4872bd0a0732383a1f57"
format_stamp: "Formatted at 2024-05-02 20:45:54 on dist-test-slave-g7g5"
I20240502 20:45:54.576457 29011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "67913548da9f4872bd0a0732383a1f57"
format_stamp: "Formatted at 2024-05-02 20:45:54 on dist-test-slave-g7g5"
I20240502 20:45:54.579478 29011 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 20:45:54.581410 29025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:54.582309 29011 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:45:54.582420 29011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal
uuid: "67913548da9f4872bd0a0732383a1f57"
format_stamp: "Formatted at 2024-05-02 20:45:54 on dist-test-slave-g7g5"
I20240502 20:45:54.582526 29011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:54.594947 29011 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:54.595459 29011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:54.595599 29011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:54.601555 29077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.126:34109 every 8 connection(s)
I20240502 20:45:54.601557 29011 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.126:34109
I20240502 20:45:54.602447 29011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 20:45:54.604802 29078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:54.605934 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 29011
I20240502 20:45:54.606050 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 20:45:54.610016 29078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Bootstrap starting.
I20240502 20:45:54.612211 29078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:54.612960 29078 log.cc:824] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Log is configured to *not* fsync() on all Append() calls
I20240502 20:45:54.614560 29078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: No bootstrap required, opened a new log
I20240502 20:45:54.616816 29078 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67913548da9f4872bd0a0732383a1f57" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 34109 } }
I20240502 20:45:54.616988 29078 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:54.617053 29078 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67913548da9f4872bd0a0732383a1f57, State: Initialized, Role: FOLLOWER
I20240502 20:45:54.617437 29078 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67913548da9f4872bd0a0732383a1f57" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 34109 } }
I20240502 20:45:54.617555 29078 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 20:45:54.617610 29078 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 20:45:54.617707 29078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:54.619060 29078 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67913548da9f4872bd0a0732383a1f57" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 34109 } }
I20240502 20:45:54.619544 29078 leader_election.cc:302] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67913548da9f4872bd0a0732383a1f57; no voters:
I20240502 20:45:54.619782 29078 leader_election.cc:288] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 20:45:54.620133 29083 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:45:54.620335 29083 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57 [term 1 LEADER]: Becoming Leader. State: Replica: 67913548da9f4872bd0a0732383a1f57, State: Running, Role: LEADER
I20240502 20:45:54.620689 29083 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67913548da9f4872bd0a0732383a1f57" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 34109 } }
I20240502 20:45:54.620780 29078 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57 [sys.catalog]: configured and running, proceeding with master startup.
I20240502 20:45:54.622143 29084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67913548da9f4872bd0a0732383a1f57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67913548da9f4872bd0a0732383a1f57" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 34109 } } }
I20240502 20:45:54.622175 29085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67913548da9f4872bd0a0732383a1f57. Latest consensus state: current_term: 1 leader_uuid: "67913548da9f4872bd0a0732383a1f57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67913548da9f4872bd0a0732383a1f57" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 34109 } } }
I20240502 20:45:54.622272 29085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57 [sys.catalog]: This master's current role is: LEADER
I20240502 20:45:54.622258 29084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57 [sys.catalog]: This master's current role is: LEADER
I20240502 20:45:54.622800 29096 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 20:45:54.625308 29096 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 20:45:54.629603 29096 catalog_manager.cc:1347] Generated new cluster ID: 168eb2dc35e64c0f8dbfbbd5ea3ff759
I20240502 20:45:54.629668 29096 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 20:45:54.658349 29096 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 20:45:54.659597 29096 catalog_manager.cc:1504] Loading token signing keys...
I20240502 20:45:54.674867 29096 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Generated new TSK 0
I20240502 20:45:54.675662 29096 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 20:45:54.679492 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.65:0
--local_ip_for_outbound_sockets=127.22.45.65
--webserver_interface=127.22.45.65
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34109
--builtin_ntp_servers=127.22.45.84:45597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 20:45:54.779934 29102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:54.780135 29102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:54.780192 29102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:54.783154 29102 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:45:54.783214 29102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:54.783298 29102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.65
I20240502 20:45:54.786796 29102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:45597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.22.45.65
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34109
--never_fsync=true
--heap_profile_path=/tmp/kudu.29102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.65
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:54.787571 29102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:54.788594 29102 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:54.795905 29110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:54.796260 29107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:54.796686 29108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:54.797065 29102 server_base.cc:1034] running on GCE node
I20240502 20:45:54.797387 29102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:54.797879 29102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:54.799019 29102 hybrid_clock.cc:648] HybridClock initialized: now 1714682754799003 us; error 27 us; skew 500 ppm
I20240502 20:45:54.800678 29102 webserver.cc:460] Webserver started at http://127.22.45.65:34427/ using document root <none> and password file <none>
I20240502 20:45:54.801177 29102 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:54.801234 29102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:54.801393 29102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:54.802953 29102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "7796ea3fba294d17a34386873969431d"
format_stamp: "Formatted at 2024-05-02 20:45:54 on dist-test-slave-g7g5"
I20240502 20:45:54.803373 29102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "7796ea3fba294d17a34386873969431d"
format_stamp: "Formatted at 2024-05-02 20:45:54 on dist-test-slave-g7g5"
I20240502 20:45:54.806334 29102 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 20:45:54.808274 29116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:54.809168 29102 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:45:54.809267 29102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "7796ea3fba294d17a34386873969431d"
format_stamp: "Formatted at 2024-05-02 20:45:54 on dist-test-slave-g7g5"
I20240502 20:45:54.809343 29102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:54.820544 29102 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:54.821149 29102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:54.821288 29102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:54.821748 29102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:54.822607 29102 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:54.822654 29102 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:54.822703 29102 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:54.822731 29102 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:54.831249 29102 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.65:43267
I20240502 20:45:54.831279 29229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.65:43267 every 8 connection(s)
I20240502 20:45:54.832053 29102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 20:45:54.834399 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 29102
I20240502 20:45:54.834550 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 20:45:54.836447 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.66:0
--local_ip_for_outbound_sockets=127.22.45.66
--webserver_interface=127.22.45.66
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34109
--builtin_ntp_servers=127.22.45.84:45597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:45:54.840164 29230 heartbeater.cc:347] Connected to a master server at 127.22.45.126:34109
I20240502 20:45:54.840349 29230 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:54.840771 29230 heartbeater.cc:511] Master 127.22.45.126:34109 requested a full tablet report, sending...
I20240502 20:45:54.842208 29042 ts_manager.cc:194] Registered new tserver with Master: 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:45:54.843892 29042 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.65:45519
W20240502 20:45:54.935560 29233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:54.935743 29233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:54.935781 29233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:54.938624 29233 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:45:54.938673 29233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:54.938742 29233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.66
I20240502 20:45:54.942070 29233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:45597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.22.45.66
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34109
--never_fsync=true
--heap_profile_path=/tmp/kudu.29233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.66
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:54.942808 29233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:54.943782 29233 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:54.951095 29238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:54.951315 29241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:54.951704 29239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:54.952131 29233 server_base.cc:1034] running on GCE node
I20240502 20:45:54.952446 29233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:54.952948 29233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:54.954087 29233 hybrid_clock.cc:648] HybridClock initialized: now 1714682754954072 us; error 30 us; skew 500 ppm
I20240502 20:45:54.955734 29233 webserver.cc:460] Webserver started at http://127.22.45.66:42261/ using document root <none> and password file <none>
I20240502 20:45:54.956229 29233 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:54.956285 29233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:54.956440 29233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:54.957926 29233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "09258ac095664fa99caf0707262cde13"
format_stamp: "Formatted at 2024-05-02 20:45:54 on dist-test-slave-g7g5"
I20240502 20:45:54.958348 29233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "09258ac095664fa99caf0707262cde13"
format_stamp: "Formatted at 2024-05-02 20:45:54 on dist-test-slave-g7g5"
I20240502 20:45:54.961309 29233 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20240502 20:45:54.963246 29247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:54.964126 29233 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:45:54.964231 29233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "09258ac095664fa99caf0707262cde13"
format_stamp: "Formatted at 2024-05-02 20:45:54 on dist-test-slave-g7g5"
I20240502 20:45:54.964308 29233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:54.976603 29233 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:54.977118 29233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:54.977252 29233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:54.977703 29233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:54.978508 29233 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:54.978554 29233 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:54.978600 29233 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:54.978628 29233 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:54.986282 29233 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.66:40945
I20240502 20:45:54.986325 29360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.66:40945 every 8 connection(s)
I20240502 20:45:54.987140 29233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 20:45:54.991022 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 29233
I20240502 20:45:54.991119 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 20:45:54.993077 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.67:0
--local_ip_for_outbound_sockets=127.22.45.67
--webserver_interface=127.22.45.67
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34109
--builtin_ntp_servers=127.22.45.84:45597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:45:54.995200 29361 heartbeater.cc:347] Connected to a master server at 127.22.45.126:34109
I20240502 20:45:54.995405 29361 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:54.995849 29361 heartbeater.cc:511] Master 127.22.45.126:34109 requested a full tablet report, sending...
I20240502 20:45:54.996783 29042 ts_manager.cc:194] Registered new tserver with Master: 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:45:54.997300 29042 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.66:48465
W20240502 20:45:55.093072 29364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:55.093253 29364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:55.093309 29364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:55.096197 29364 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:45:55.096256 29364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:55.096334 29364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.67
I20240502 20:45:55.099730 29364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:45597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.22.45.67
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34109
--never_fsync=true
--heap_profile_path=/tmp/kudu.29364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.67
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:55.100454 29364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:55.101438 29364 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:55.108901 29369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:55.109200 29370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:55.109525 29372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:55.109934 29364 server_base.cc:1034] running on GCE node
I20240502 20:45:55.110244 29364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:55.110793 29364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:55.111940 29364 hybrid_clock.cc:648] HybridClock initialized: now 1714682755111920 us; error 38 us; skew 500 ppm
I20240502 20:45:55.113790 29364 webserver.cc:460] Webserver started at http://127.22.45.67:41639/ using document root <none> and password file <none>
I20240502 20:45:55.114279 29364 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:55.114337 29364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:55.114554 29364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:55.116077 29364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
format_stamp: "Formatted at 2024-05-02 20:45:55 on dist-test-slave-g7g5"
I20240502 20:45:55.116530 29364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
format_stamp: "Formatted at 2024-05-02 20:45:55 on dist-test-slave-g7g5"
I20240502 20:45:55.119632 29364 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 20:45:55.121734 29378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:55.122716 29364 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 20:45:55.122836 29364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
format_stamp: "Formatted at 2024-05-02 20:45:55 on dist-test-slave-g7g5"
I20240502 20:45:55.122922 29364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:55.153856 29364 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:55.154431 29364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:55.154598 29364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:55.155069 29364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:55.155882 29364 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:55.155928 29364 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:55.155964 29364 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:55.155992 29364 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:55.163726 29364 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.67:44095
I20240502 20:45:55.163753 29491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.67:44095 every 8 connection(s)
I20240502 20:45:55.164561 29364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 20:45:55.168879 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 29364
I20240502 20:45:55.168967 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 20:45:55.171099 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.68:0
--local_ip_for_outbound_sockets=127.22.45.68
--webserver_interface=127.22.45.68
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34109
--builtin_ntp_servers=127.22.45.84:45597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:45:55.172806 29492 heartbeater.cc:347] Connected to a master server at 127.22.45.126:34109
I20240502 20:45:55.172991 29492 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:55.173375 29492 heartbeater.cc:511] Master 127.22.45.126:34109 requested a full tablet report, sending...
I20240502 20:45:55.174304 29042 ts_manager.cc:194] Registered new tserver with Master: dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:45:55.175081 29042 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.67:58439
W20240502 20:45:55.270493 29495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:55.270669 29495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:55.270710 29495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:55.273582 29495 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:45:55.273635 29495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:55.273699 29495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.68
I20240502 20:45:55.277050 29495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:45597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.22.45.68
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34109
--never_fsync=true
--heap_profile_path=/tmp/kudu.29495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.68
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:55.277971 29495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:55.279072 29495 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:55.286300 29500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:55.286681 29503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:55.287061 29501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:55.287487 29495 server_base.cc:1034] running on GCE node
I20240502 20:45:55.287791 29495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:55.288249 29495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:55.289391 29495 hybrid_clock.cc:648] HybridClock initialized: now 1714682755289378 us; error 30 us; skew 500 ppm
I20240502 20:45:55.291093 29495 webserver.cc:460] Webserver started at http://127.22.45.68:33287/ using document root <none> and password file <none>
I20240502 20:45:55.291575 29495 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:55.291630 29495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:55.291785 29495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:55.293270 29495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "db17f26a73f3447b92aa9a947e3a7057"
format_stamp: "Formatted at 2024-05-02 20:45:55 on dist-test-slave-g7g5"
I20240502 20:45:55.293695 29495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "db17f26a73f3447b92aa9a947e3a7057"
format_stamp: "Formatted at 2024-05-02 20:45:55 on dist-test-slave-g7g5"
I20240502 20:45:55.296701 29495 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20240502 20:45:55.298612 29509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:55.299487 29495 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20240502 20:45:55.299584 29495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "db17f26a73f3447b92aa9a947e3a7057"
format_stamp: "Formatted at 2024-05-02 20:45:55 on dist-test-slave-g7g5"
I20240502 20:45:55.299659 29495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:55.322068 29495 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:55.322649 29495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:55.322782 29495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:55.323246 29495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:55.324043 29495 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:55.324087 29495 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:55.324136 29495 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:55.324160 29495 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:55.331863 29495 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.68:45897
I20240502 20:45:55.331885 29622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.68:45897 every 8 connection(s)
I20240502 20:45:55.332672 29495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 20:45:55.336074 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 29495
I20240502 20:45:55.336179 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 20:45:55.338102 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.69:0
--local_ip_for_outbound_sockets=127.22.45.69
--webserver_interface=127.22.45.69
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34109
--builtin_ntp_servers=127.22.45.84:45597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:45:55.341102 29623 heartbeater.cc:347] Connected to a master server at 127.22.45.126:34109
I20240502 20:45:55.341302 29623 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:55.341711 29623 heartbeater.cc:511] Master 127.22.45.126:34109 requested a full tablet report, sending...
I20240502 20:45:55.342546 29042 ts_manager.cc:194] Registered new tserver with Master: db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:45:55.343065 29042 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.68:37873
W20240502 20:45:55.438212 29626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:55.438395 29626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:55.438439 29626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:55.441298 29626 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:45:55.441352 29626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:55.441440 29626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.69
I20240502 20:45:55.444878 29626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:45597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.69:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.22.45.69
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34109
--never_fsync=true
--heap_profile_path=/tmp/kudu.29626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.69
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:55.445621 29626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:55.446630 29626 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:55.453775 29632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:55.454142 29634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:55.454527 29631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:55.454900 29626 server_base.cc:1034] running on GCE node
I20240502 20:45:55.455199 29626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:55.455667 29626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:55.456804 29626 hybrid_clock.cc:648] HybridClock initialized: now 1714682755456784 us; error 33 us; skew 500 ppm
I20240502 20:45:55.458420 29626 webserver.cc:460] Webserver started at http://127.22.45.69:46521/ using document root <none> and password file <none>
I20240502 20:45:55.458936 29626 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:55.458989 29626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:55.459151 29626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:55.460623 29626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
format_stamp: "Formatted at 2024-05-02 20:45:55 on dist-test-slave-g7g5"
I20240502 20:45:55.461048 29626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
format_stamp: "Formatted at 2024-05-02 20:45:55 on dist-test-slave-g7g5"
I20240502 20:45:55.464066 29626 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20240502 20:45:55.465926 29640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:55.466825 29626 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:45:55.466936 29626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
format_stamp: "Formatted at 2024-05-02 20:45:55 on dist-test-slave-g7g5"
I20240502 20:45:55.467011 29626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:55.498705 29626 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:55.499254 29626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:55.499393 29626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:55.499874 29626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:55.500679 29626 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:55.500727 29626 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:55.500777 29626 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:55.500802 29626 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:55.508517 29626 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.69:43887
I20240502 20:45:55.508538 29753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.69:43887 every 8 connection(s)
I20240502 20:45:55.509331 29626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 20:45:55.513592 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 29626
I20240502 20:45:55.513710 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240502 20:45:55.515897 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.70:0
--local_ip_for_outbound_sockets=127.22.45.70
--webserver_interface=127.22.45.70
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34109
--builtin_ntp_servers=127.22.45.84:45597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:45:55.517525 29754 heartbeater.cc:347] Connected to a master server at 127.22.45.126:34109
I20240502 20:45:55.517745 29754 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:55.518165 29754 heartbeater.cc:511] Master 127.22.45.126:34109 requested a full tablet report, sending...
I20240502 20:45:55.519011 29042 ts_manager.cc:194] Registered new tserver with Master: 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:45:55.519524 29042 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.69:42089
W20240502 20:45:55.617107 29757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:55.617282 29757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:55.617322 29757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:55.620172 29757 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:45:55.620224 29757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:55.620290 29757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.70
I20240502 20:45:55.623751 29757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:45597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.70:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.22.45.70
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34109
--never_fsync=true
--heap_profile_path=/tmp/kudu.29757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.70
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:55.624430 29757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:55.625385 29757 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:55.632716 29762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:55.633111 29765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:55.633452 29763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:55.633939 29757 server_base.cc:1034] running on GCE node
I20240502 20:45:55.634263 29757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:55.634865 29757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:55.636008 29757 hybrid_clock.cc:648] HybridClock initialized: now 1714682755635990 us; error 28 us; skew 500 ppm
I20240502 20:45:55.637626 29757 webserver.cc:460] Webserver started at http://127.22.45.70:35339/ using document root <none> and password file <none>
I20240502 20:45:55.638125 29757 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:55.638182 29757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:55.638336 29757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:55.639847 29757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "315915efcf2b413ca7f9d874338e1b2d"
format_stamp: "Formatted at 2024-05-02 20:45:55 on dist-test-slave-g7g5"
I20240502 20:45:55.640267 29757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "315915efcf2b413ca7f9d874338e1b2d"
format_stamp: "Formatted at 2024-05-02 20:45:55 on dist-test-slave-g7g5"
I20240502 20:45:55.643270 29757 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 20:45:55.645254 29771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:55.646190 29757 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:45:55.646287 29757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "315915efcf2b413ca7f9d874338e1b2d"
format_stamp: "Formatted at 2024-05-02 20:45:55 on dist-test-slave-g7g5"
I20240502 20:45:55.646366 29757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:55.678815 29757 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:55.679384 29757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:55.679520 29757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:55.680003 29757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:55.680819 29757 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:55.680863 29757 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:55.680912 29757 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:55.680938 29757 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:55.688728 29757 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.70:46751
I20240502 20:45:55.688763 29884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.70:46751 every 8 connection(s)
I20240502 20:45:55.689554 29757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 20:45:55.691886 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 29757
I20240502 20:45:55.692019 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240502 20:45:55.693997 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.71:0
--local_ip_for_outbound_sockets=127.22.45.71
--webserver_interface=127.22.45.71
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34109
--builtin_ntp_servers=127.22.45.84:45597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:45:55.697899 29885 heartbeater.cc:347] Connected to a master server at 127.22.45.126:34109
I20240502 20:45:55.698086 29885 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:55.698500 29885 heartbeater.cc:511] Master 127.22.45.126:34109 requested a full tablet report, sending...
I20240502 20:45:55.699434 29042 ts_manager.cc:194] Registered new tserver with Master: 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:45:55.700131 29042 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.70:40463
W20240502 20:45:55.794243 29888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:55.794437 29888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:55.794505 29888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:55.797386 29888 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:45:55.797441 29888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:55.797518 29888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.71
I20240502 20:45:55.800956 29888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:45597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.71:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.22.45.71
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34109
--never_fsync=true
--heap_profile_path=/tmp/kudu.29888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.71
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:55.801721 29888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:55.802752 29888 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:55.809978 29896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:55.810328 29893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:55.810712 29894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:55.811156 29888 server_base.cc:1034] running on GCE node
I20240502 20:45:55.811475 29888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:55.811962 29888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:55.813098 29888 hybrid_clock.cc:648] HybridClock initialized: now 1714682755813090 us; error 26 us; skew 500 ppm
I20240502 20:45:55.814731 29888 webserver.cc:460] Webserver started at http://127.22.45.71:45465/ using document root <none> and password file <none>
I20240502 20:45:55.815220 29888 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:55.815276 29888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:55.815430 29888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:55.816900 29888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
format_stamp: "Formatted at 2024-05-02 20:45:55 on dist-test-slave-g7g5"
I20240502 20:45:55.817312 29888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
format_stamp: "Formatted at 2024-05-02 20:45:55 on dist-test-slave-g7g5"
I20240502 20:45:55.820302 29888 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 20:45:55.822321 29902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:55.823273 29888 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:45:55.823379 29888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
format_stamp: "Formatted at 2024-05-02 20:45:55 on dist-test-slave-g7g5"
I20240502 20:45:55.823454 29888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:55.836236 29888 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:55.836745 29888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:55.836879 29888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:55.837328 29888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:55.838115 29888 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:55.838162 29888 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:55.838209 29888 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:55.838235 29888 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:55.845986 29888 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.71:36787
I20240502 20:45:55.846024 30015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.71:36787 every 8 connection(s)
I20240502 20:45:55.846050 29230 heartbeater.cc:503] Master 127.22.45.126:34109 was elected leader, sending a full tablet report...
I20240502 20:45:55.846894 29888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 20:45:55.849467 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 29888
I20240502 20:45:55.849560 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240502 20:45:55.851559 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.72:0
--local_ip_for_outbound_sockets=127.22.45.72
--webserver_interface=127.22.45.72
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34109
--builtin_ntp_servers=127.22.45.84:45597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:45:55.855262 30016 heartbeater.cc:347] Connected to a master server at 127.22.45.126:34109
I20240502 20:45:55.855445 30016 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:55.855823 30016 heartbeater.cc:511] Master 127.22.45.126:34109 requested a full tablet report, sending...
I20240502 20:45:55.856670 29042 ts_manager.cc:194] Registered new tserver with Master: e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:45:55.857167 29042 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.71:42055
W20240502 20:45:55.951491 30019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:45:55.951675 30019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:45:55.951718 30019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:45:55.954619 30019 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:45:55.954671 30019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:45:55.954759 30019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.72
I20240502 20:45:55.958117 30019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:45597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.72:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.22.45.72
--webserver_port=0
--tserver_master_addrs=127.22.45.126:34109
--never_fsync=true
--heap_profile_path=/tmp/kudu.30019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.72
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:45:55.958890 30019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:45:55.959880 30019 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:45:55.967093 30024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:55.967466 30027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:45:55.967921 30025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:45:55.968302 30019 server_base.cc:1034] running on GCE node
I20240502 20:45:55.968621 30019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:45:55.969074 30019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:45:55.970211 30019 hybrid_clock.cc:648] HybridClock initialized: now 1714682755970201 us; error 25 us; skew 500 ppm
I20240502 20:45:55.971925 30019 webserver.cc:460] Webserver started at http://127.22.45.72:38267/ using document root <none> and password file <none>
I20240502 20:45:55.972407 30019 fs_manager.cc:359] Metadata directory not provided
I20240502 20:45:55.972476 30019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:45:55.972635 30019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:45:55.974117 30019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "e6b60803d0764d228c863e2da65e3691"
format_stamp: "Formatted at 2024-05-02 20:45:55 on dist-test-slave-g7g5"
I20240502 20:45:55.974572 30019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "e6b60803d0764d228c863e2da65e3691"
format_stamp: "Formatted at 2024-05-02 20:45:55 on dist-test-slave-g7g5"
I20240502 20:45:55.977563 30019 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240502 20:45:55.979526 30033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:55.980410 30019 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 20:45:55.980507 30019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "e6b60803d0764d228c863e2da65e3691"
format_stamp: "Formatted at 2024-05-02 20:45:55 on dist-test-slave-g7g5"
I20240502 20:45:55.980582 30019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:45:55.991341 30019 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:45:55.991847 30019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:45:55.991976 30019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:45:55.992416 30019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:45:55.993221 30019 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:45:55.993266 30019 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:55.993310 30019 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:45:55.993338 30019 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:45:55.999368 29361 heartbeater.cc:503] Master 127.22.45.126:34109 was elected leader, sending a full tablet report...
I20240502 20:45:56.001399 30019 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.72:35207
I20240502 20:45:56.001426 30146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.72:35207 every 8 connection(s)
I20240502 20:45:56.002207 30019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 20:45:56.006397 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 30019
I20240502 20:45:56.006527 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240502 20:45:56.012557 30147 heartbeater.cc:347] Connected to a master server at 127.22.45.126:34109
I20240502 20:45:56.012770 30147 heartbeater.cc:464] Registering TS with master...
I20240502 20:45:56.013242 30147 heartbeater.cc:511] Master 127.22.45.126:34109 requested a full tablet report, sending...
I20240502 20:45:56.014217 29042 ts_manager.cc:194] Registered new tserver with Master: e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:45:56.014746 29042 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.72:53405
I20240502 20:45:56.024291 22709 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240502 20:45:56.032837 22709 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240502 20:45:56.032904 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 29626
I20240502 20:45:56.037575 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 29757
I20240502 20:45:56.042074 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 29888
I20240502 20:45:56.046500 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 30019
I20240502 20:45:56.176863 29492 heartbeater.cc:503] Master 127.22.45.126:34109 was elected leader, sending a full tablet report...
I20240502 20:45:56.344795 29623 heartbeater.cc:503] Master 127.22.45.126:34109 was elected leader, sending a full tablet report...
I20240502 20:45:59.801996 29042 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:45:59.821339 29425 tablet_service.cc:1467] Processing CreateTablet for tablet 8912e9b8df214e2cb7688860793f1ab7 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0ccbd3f0bec843cb859a6fc61e0ec2a9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:59.821851 29163 tablet_service.cc:1467] Processing CreateTablet for tablet a902eaf8bf274fb582550c1cf61d9465 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0ccbd3f0bec843cb859a6fc61e0ec2a9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:59.821923 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 5e6ae7bb0e174bd5bf8ca352bbf9d694 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0ccbd3f0bec843cb859a6fc61e0ec2a9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:59.821851 29164 tablet_service.cc:1467] Processing CreateTablet for tablet 5e6ae7bb0e174bd5bf8ca352bbf9d694 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0ccbd3f0bec843cb859a6fc61e0ec2a9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:59.822314 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8912e9b8df214e2cb7688860793f1ab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:59.822770 29294 tablet_service.cc:1467] Processing CreateTablet for tablet a902eaf8bf274fb582550c1cf61d9465 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0ccbd3f0bec843cb859a6fc61e0ec2a9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:59.822827 29164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e6ae7bb0e174bd5bf8ca352bbf9d694. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:59.821342 29426 tablet_service.cc:1467] Processing CreateTablet for tablet 5e6ae7bb0e174bd5bf8ca352bbf9d694 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0ccbd3f0bec843cb859a6fc61e0ec2a9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:59.823181 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a902eaf8bf274fb582550c1cf61d9465. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:59.823202 29426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e6ae7bb0e174bd5bf8ca352bbf9d694. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:59.823966 29162 tablet_service.cc:1467] Processing CreateTablet for tablet 8912e9b8df214e2cb7688860793f1ab7 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0ccbd3f0bec843cb859a6fc61e0ec2a9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:59.825860 29556 tablet_service.cc:1467] Processing CreateTablet for tablet 8912e9b8df214e2cb7688860793f1ab7 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0ccbd3f0bec843cb859a6fc61e0ec2a9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:59.826714 29557 tablet_service.cc:1467] Processing CreateTablet for tablet a902eaf8bf274fb582550c1cf61d9465 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0ccbd3f0bec843cb859a6fc61e0ec2a9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:45:59.827088 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a902eaf8bf274fb582550c1cf61d9465. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:59.828934 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a902eaf8bf274fb582550c1cf61d9465. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:59.829066 30169 tablet_bootstrap.cc:492] T a902eaf8bf274fb582550c1cf61d9465 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:45:59.829370 30167 tablet_bootstrap.cc:492] T 8912e9b8df214e2cb7688860793f1ab7 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:45:59.830488 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8912e9b8df214e2cb7688860793f1ab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:59.831162 30169 tablet_bootstrap.cc:654] T a902eaf8bf274fb582550c1cf61d9465 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:59.831162 30167 tablet_bootstrap.cc:654] T 8912e9b8df214e2cb7688860793f1ab7 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:59.831949 30167 log.cc:824] T 8912e9b8df214e2cb7688860793f1ab7 P dbcdc6ac635b44b185c3eba63f66e9d6: Log is configured to *not* fsync() on all Append() calls
I20240502 20:45:59.831949 30169 log.cc:824] T a902eaf8bf274fb582550c1cf61d9465 P 09258ac095664fa99caf0707262cde13: Log is configured to *not* fsync() on all Append() calls
I20240502 20:45:59.832607 30168 tablet_bootstrap.cc:492] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:45:59.833714 30167 tablet_bootstrap.cc:492] T 8912e9b8df214e2cb7688860793f1ab7 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:45:59.833832 30169 tablet_bootstrap.cc:492] T a902eaf8bf274fb582550c1cf61d9465 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:45:59.833933 30167 ts_tablet_manager.cc:1367] T 8912e9b8df214e2cb7688860793f1ab7 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240502 20:45:59.834007 30169 ts_tablet_manager.cc:1367] T a902eaf8bf274fb582550c1cf61d9465 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240502 20:45:59.834432 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8912e9b8df214e2cb7688860793f1ab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:59.835350 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e6ae7bb0e174bd5bf8ca352bbf9d694. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:45:59.837755 30167 raft_consensus.cc:348] T 8912e9b8df214e2cb7688860793f1ab7 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:45:59.837858 30169 raft_consensus.cc:348] T a902eaf8bf274fb582550c1cf61d9465 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:45:59.837965 30167 raft_consensus.cc:374] T 8912e9b8df214e2cb7688860793f1ab7 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:59.838011 30167 raft_consensus.cc:729] T 8912e9b8df214e2cb7688860793f1ab7 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:45:59.838472 30167 consensus_queue.cc:260] T 8912e9b8df214e2cb7688860793f1ab7 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:45:59.839148 30168 tablet_bootstrap.cc:654] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:59.839419 30167 ts_tablet_manager.cc:1398] T 8912e9b8df214e2cb7688860793f1ab7 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240502 20:45:59.839794 30167 tablet_bootstrap.cc:492] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:45:59.839915 30168 log.cc:824] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 7796ea3fba294d17a34386873969431d: Log is configured to *not* fsync() on all Append() calls
I20240502 20:45:59.840044 30169 raft_consensus.cc:374] T a902eaf8bf274fb582550c1cf61d9465 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:59.840147 30169 raft_consensus.cc:729] T a902eaf8bf274fb582550c1cf61d9465 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:45:59.840705 30167 tablet_bootstrap.cc:654] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:59.840629 30169 consensus_queue.cc:260] T a902eaf8bf274fb582550c1cf61d9465 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:45:59.841434 30169 ts_tablet_manager.cc:1398] T a902eaf8bf274fb582550c1cf61d9465 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20240502 20:45:59.841753 30169 tablet_bootstrap.cc:492] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:45:59.842932 30169 tablet_bootstrap.cc:654] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:59.843473 30167 tablet_bootstrap.cc:492] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:45:59.843559 30167 ts_tablet_manager.cc:1367] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:45:59.843669 30175 tablet_bootstrap.cc:492] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:45:59.844345 30167 raft_consensus.cc:348] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:45:59.844445 30167 raft_consensus.cc:374] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:59.844483 30167 raft_consensus.cc:729] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:45:59.844595 30167 consensus_queue.cc:260] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:45:59.844838 30167 ts_tablet_manager.cc:1398] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:45:59.845181 30173 raft_consensus.cc:482] T a902eaf8bf274fb582550c1cf61d9465 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:45:59.845219 30168 tablet_bootstrap.cc:492] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:45:59.845304 30173 raft_consensus.cc:504] T a902eaf8bf274fb582550c1cf61d9465 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:45:59.845367 30168 ts_tablet_manager.cc:1367] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20240502 20:45:59.845669 30175 tablet_bootstrap.cc:654] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:59.846941 30175 log.cc:824] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057: Log is configured to *not* fsync() on all Append() calls
I20240502 20:45:59.846964 30172 raft_consensus.cc:482] T 8912e9b8df214e2cb7688860793f1ab7 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:45:59.847081 30172 raft_consensus.cc:504] T 8912e9b8df214e2cb7688860793f1ab7 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:45:59.849246 30172 leader_election.cc:288] T 8912e9b8df214e2cb7688860793f1ab7 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:45:59.849942 30169 tablet_bootstrap.cc:492] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:45:59.850024 30169 ts_tablet_manager.cc:1367] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 20:45:59.850263 30173 leader_election.cc:288] T a902eaf8bf274fb582550c1cf61d9465 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:45:59.850536 30169 raft_consensus.cc:348] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:45:59.850625 30169 raft_consensus.cc:374] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:59.850651 30169 raft_consensus.cc:729] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:45:59.850764 30169 consensus_queue.cc:260] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:45:59.850872 30173 raft_consensus.cc:482] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:45:59.850945 30173 raft_consensus.cc:504] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:45:59.851003 30169 ts_tablet_manager.cc:1398] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:45:59.850993 30168 raft_consensus.cc:348] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:45:59.851464 30168 raft_consensus.cc:374] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:59.851532 30168 raft_consensus.cc:729] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:45:59.851876 30173 leader_election.cc:288] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:45:59.851912 30168 consensus_queue.cc:260] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:45:59.853106 30168 ts_tablet_manager.cc:1398] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20240502 20:45:59.853420 30168 tablet_bootstrap.cc:492] T a902eaf8bf274fb582550c1cf61d9465 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:45:59.854264 30168 tablet_bootstrap.cc:654] T a902eaf8bf274fb582550c1cf61d9465 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:59.857506 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8912e9b8df214e2cb7688860793f1ab7" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
W20240502 20:45:59.858220 29382 leader_election.cc:341] T 8912e9b8df214e2cb7688860793f1ab7 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): Illegal state: must be running to vote when last-logged opid is not known
I20240502 20:45:59.858731 29184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6ae7bb0e174bd5bf8ca352bbf9d694" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:45:59.858922 29184 raft_consensus.cc:2454] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:45:59.858150 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8912e9b8df214e2cb7688860793f1ab7" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:45:59.859287 29251 leader_election.cc:302] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
W20240502 20:45:59.859584 29380 leader_election.cc:341] T 8912e9b8df214e2cb7688860793f1ab7 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): Illegal state: must be running to vote when last-logged opid is not known
I20240502 20:45:59.859586 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a902eaf8bf274fb582550c1cf61d9465" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:45:59.859668 29380 leader_election.cc:302] T 8912e9b8df214e2cb7688860793f1ab7 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbcdc6ac635b44b185c3eba63f66e9d6; no voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057
I20240502 20:45:59.859946 30175 tablet_bootstrap.cc:492] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
W20240502 20:45:59.859956 29251 leader_election.cc:341] T a902eaf8bf274fb582550c1cf61d9465 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): Illegal state: must be running to vote when last-logged opid is not known
I20240502 20:45:59.860040 30173 raft_consensus.cc:2781] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:45:59.860107 30173 raft_consensus.cc:482] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:45:59.860123 30175 ts_tablet_manager.cc:1367] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.000s
I20240502 20:45:59.860155 30173 raft_consensus.cc:3037] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:59.860164 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a902eaf8bf274fb582550c1cf61d9465" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
W20240502 20:45:59.860826 29249 leader_election.cc:341] T a902eaf8bf274fb582550c1cf61d9465 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): Illegal state: must be running to vote when last-logged opid is not known
I20240502 20:45:59.860898 29249 leader_election.cc:302] T a902eaf8bf274fb582550c1cf61d9465 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 09258ac095664fa99caf0707262cde13; no voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057
I20240502 20:45:59.861083 30173 raft_consensus.cc:504] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:45:59.861407 30173 leader_election.cc:288] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:45:59.861490 30173 raft_consensus.cc:2726] T a902eaf8bf274fb582550c1cf61d9465 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 20:45:59.861557 30168 tablet_bootstrap.cc:492] T a902eaf8bf274fb582550c1cf61d9465 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:45:59.861639 30168 ts_tablet_manager.cc:1367] T a902eaf8bf274fb582550c1cf61d9465 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20240502 20:45:59.861618 30172 raft_consensus.cc:2726] T 8912e9b8df214e2cb7688860793f1ab7 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 20:45:59.862035 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6ae7bb0e174bd5bf8ca352bbf9d694" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:45:59.862141 29175 raft_consensus.cc:3037] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:59.862373 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6ae7bb0e174bd5bf8ca352bbf9d694" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:45:59.862448 30175 raft_consensus.cc:348] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:45:59.862639 30175 raft_consensus.cc:374] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:59.862694 30175 raft_consensus.cc:729] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:45:59.862380 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6ae7bb0e174bd5bf8ca352bbf9d694" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:45:59.863082 29446 raft_consensus.cc:2454] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:45:59.863057 30175 consensus_queue.cc:260] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:45:59.863171 29175 raft_consensus.cc:2454] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:45:59.863503 29251 leader_election.cc:302] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:45:59.863554 30168 raft_consensus.cc:348] T a902eaf8bf274fb582550c1cf61d9465 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:45:59.863649 30168 raft_consensus.cc:374] T a902eaf8bf274fb582550c1cf61d9465 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:59.863690 30168 raft_consensus.cc:729] T a902eaf8bf274fb582550c1cf61d9465 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:45:59.863809 30175 ts_tablet_manager.cc:1398] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20240502 20:45:59.863817 30168 consensus_queue.cc:260] T a902eaf8bf274fb582550c1cf61d9465 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:45:59.863956 30173 raft_consensus.cc:2781] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:45:59.864034 30173 raft_consensus.cc:686] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:45:59.864086 30168 ts_tablet_manager.cc:1398] T a902eaf8bf274fb582550c1cf61d9465 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:45:59.864228 30168 tablet_bootstrap.cc:492] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:45:59.864274 30173 consensus_queue.cc:237] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:45:59.864877 30175 tablet_bootstrap.cc:492] T 8912e9b8df214e2cb7688860793f1ab7 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:45:59.865783 30175 tablet_bootstrap.cc:654] T 8912e9b8df214e2cb7688860793f1ab7 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:59.866401 30168 tablet_bootstrap.cc:654] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:45:59.866812 29041 catalog_manager.cc:5484] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:45:59.867578 30168 tablet_bootstrap.cc:492] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:45:59.867658 30168 ts_tablet_manager.cc:1367] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 20:45:59.867959 30175 tablet_bootstrap.cc:492] T 8912e9b8df214e2cb7688860793f1ab7 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:45:59.868042 30175 ts_tablet_manager.cc:1367] T 8912e9b8df214e2cb7688860793f1ab7 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 20:45:59.868232 30168 raft_consensus.cc:348] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:45:59.868341 30168 raft_consensus.cc:374] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:59.868379 30168 raft_consensus.cc:729] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:45:59.868492 30168 consensus_queue.cc:260] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:45:59.868731 30168 ts_tablet_manager.cc:1398] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:45:59.869006 30175 raft_consensus.cc:348] T 8912e9b8df214e2cb7688860793f1ab7 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:45:59.869104 30175 raft_consensus.cc:374] T 8912e9b8df214e2cb7688860793f1ab7 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:45:59.869139 30175 raft_consensus.cc:729] T 8912e9b8df214e2cb7688860793f1ab7 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:45:59.869391 30175 consensus_queue.cc:260] T 8912e9b8df214e2cb7688860793f1ab7 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:45:59.869721 30175 ts_tablet_manager.cc:1398] T 8912e9b8df214e2cb7688860793f1ab7 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
W20240502 20:45:59.918241 29493 tablet.cc:2369] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P dbcdc6ac635b44b185c3eba63f66e9d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 20:45:59.980120 30187 raft_consensus.cc:482] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:45:59.980247 30187 raft_consensus.cc:504] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:45:59.981312 30187 leader_election.cc:288] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:45:59.984665 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8912e9b8df214e2cb7688860793f1ab7" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:45:59.984666 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8912e9b8df214e2cb7688860793f1ab7" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:45:59.984808 29445 raft_consensus.cc:2454] T 8912e9b8df214e2cb7688860793f1ab7 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:45:59.984897 29577 raft_consensus.cc:2454] T 8912e9b8df214e2cb7688860793f1ab7 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:45:59.985137 29119 leader_election.cc:302] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:45:59.985401 30187 raft_consensus.cc:2781] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:45:59.985463 30187 raft_consensus.cc:482] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:45:59.985508 30187 raft_consensus.cc:3037] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:59.986286 30187 raft_consensus.cc:504] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:45:59.986631 30187 leader_election.cc:288] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:45:59.986912 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8912e9b8df214e2cb7688860793f1ab7" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:45:59.986948 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8912e9b8df214e2cb7688860793f1ab7" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:45:59.987027 29577 raft_consensus.cc:3037] T 8912e9b8df214e2cb7688860793f1ab7 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:59.987054 29445 raft_consensus.cc:3037] T 8912e9b8df214e2cb7688860793f1ab7 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:45:59.987787 29445 raft_consensus.cc:2454] T 8912e9b8df214e2cb7688860793f1ab7 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:45:59.988027 29577 raft_consensus.cc:2454] T 8912e9b8df214e2cb7688860793f1ab7 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:45:59.988041 29119 leader_election.cc:302] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:45:59.988404 30187 raft_consensus.cc:2781] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:45:59.988592 30187 raft_consensus.cc:686] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:45:59.988880 30187 consensus_queue.cc:237] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
W20240502 20:45:59.991189 29362 tablet.cc:2369] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 20:45:59.991434 29041 catalog_manager.cc:5484] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:00.062690 30189 raft_consensus.cc:482] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:00.062835 30189 raft_consensus.cc:504] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:00.063855 30189 leader_election.cc:288] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:00.067368 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a902eaf8bf274fb582550c1cf61d9465" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:00.067523 29175 raft_consensus.cc:2454] T a902eaf8bf274fb582550c1cf61d9465 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:00.067515 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a902eaf8bf274fb582550c1cf61d9465" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:00.067747 29315 raft_consensus.cc:2454] T a902eaf8bf274fb582550c1cf61d9465 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:00.067836 29513 leader_election.cc:302] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:00.068065 30189 raft_consensus.cc:2781] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:00.068128 30189 raft_consensus.cc:482] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:00.068154 30189 raft_consensus.cc:3037] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:00.069000 30189 raft_consensus.cc:504] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:00.069311 30189 leader_election.cc:288] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:00.069607 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a902eaf8bf274fb582550c1cf61d9465" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:00.069649 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a902eaf8bf274fb582550c1cf61d9465" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:00.069708 29175 raft_consensus.cc:3037] T a902eaf8bf274fb582550c1cf61d9465 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:00.069743 29315 raft_consensus.cc:3037] T a902eaf8bf274fb582550c1cf61d9465 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:00.070412 29175 raft_consensus.cc:2454] T a902eaf8bf274fb582550c1cf61d9465 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:00.070544 29315 raft_consensus.cc:2454] T a902eaf8bf274fb582550c1cf61d9465 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:00.070683 29513 leader_election.cc:302] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:00.070858 30189 raft_consensus.cc:2781] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:00.071048 30189 raft_consensus.cc:686] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Becoming Leader. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Running, Role: LEADER
I20240502 20:46:00.071318 30189 consensus_queue.cc:237] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:00.073961 29041 catalog_manager.cc:5484] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: term changed from 0 to 1, leader changed from <none> to db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
W20240502 20:46:00.086452 29624 tablet.cc:2369] T 8912e9b8df214e2cb7688860793f1ab7 P db17f26a73f3447b92aa9a947e3a7057: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 20:46:00.086452 29231 tablet.cc:2369] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 7796ea3fba294d17a34386873969431d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 20:46:00.205771 30201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:00.205981 30201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:00.208969 30201 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20240502 20:46:00.293233 30178 consensus_queue.cc:1035] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:00.296649 29445 raft_consensus.cc:3037] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:00.298327 30173 consensus_queue.cc:1035] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:00.299751 29175 raft_consensus.cc:1261] T a902eaf8bf274fb582550c1cf61d9465 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:00.299906 29315 raft_consensus.cc:1261] T a902eaf8bf274fb582550c1cf61d9465 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:00.300312 29577 raft_consensus.cc:1261] T 8912e9b8df214e2cb7688860793f1ab7 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:00.300752 30199 consensus_queue.cc:1035] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:00.300829 30187 consensus_queue.cc:1035] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:00.301116 30199 consensus_queue.cc:1035] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:00.306789 29445 raft_consensus.cc:1261] T 8912e9b8df214e2cb7688860793f1ab7 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:00.307405 30187 consensus_queue.cc:1035] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:00.312731 30232 mvcc.cc:205] Tried to move back new op lower bound from 7023340586227847168 to 7023340584406528000. Current Snapshot: MvccSnapshot[applied={T|T < 7023340586205032448}]
I20240502 20:46:00.313522 30234 mvcc.cc:205] Tried to move back new op lower bound from 7023340586180870144 to 7023340584916877312. Current Snapshot: MvccSnapshot[applied={T|T < 7023340586180870144}]
I20240502 20:46:00.313657 30231 mvcc.cc:205] Tried to move back new op lower bound from 7023340586180870144 to 7023340584916877312. Current Snapshot: MvccSnapshot[applied={T|T < 7023340586180870144}]
I20240502 20:46:00.330281 30239 mvcc.cc:205] Tried to move back new op lower bound from 7023340586198077440 to 7023340584916877312. Current Snapshot: MvccSnapshot[applied={T|T < 7023340586180870144}]
INSERT report
rows total: 2000
time total: 120.905 ms
time per row: 0.0604523 ms
I20240502 20:46:00.367734 29040 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:00.372681 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 689b7368353a430dbffa1cf44f9b5949 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8011864bab7b4506a89a8bbb019d244f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:00.372749 29426 tablet_service.cc:1467] Processing CreateTablet for tablet 689b7368353a430dbffa1cf44f9b5949 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8011864bab7b4506a89a8bbb019d244f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:00.372749 29556 tablet_service.cc:1467] Processing CreateTablet for tablet 689b7368353a430dbffa1cf44f9b5949 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8011864bab7b4506a89a8bbb019d244f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:00.372982 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 689b7368353a430dbffa1cf44f9b5949. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:00.373006 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 689b7368353a430dbffa1cf44f9b5949. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:00.373006 29426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 689b7368353a430dbffa1cf44f9b5949. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:00.374703 30261 tablet_bootstrap.cc:492] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:00.375419 29162 tablet_service.cc:1467] Processing CreateTablet for tablet 7fd046a0f0004e5fa6e0825cb626c2ff (DEFAULT_TABLE table=rebalance_test_table_1 [id=8011864bab7b4506a89a8bbb019d244f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:00.375533 30261 tablet_bootstrap.cc:654] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:00.375677 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fd046a0f0004e5fa6e0825cb626c2ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:00.375788 29425 tablet_service.cc:1467] Processing CreateTablet for tablet 7fd046a0f0004e5fa6e0825cb626c2ff (DEFAULT_TABLE table=rebalance_test_table_1 [id=8011864bab7b4506a89a8bbb019d244f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:00.376003 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fd046a0f0004e5fa6e0825cb626c2ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:00.376094 30262 tablet_bootstrap.cc:492] T 689b7368353a430dbffa1cf44f9b5949 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:00.377008 30262 tablet_bootstrap.cc:654] T 689b7368353a430dbffa1cf44f9b5949 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:00.377441 30265 tablet_bootstrap.cc:492] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:00.377660 30263 tablet_bootstrap.cc:492] T 689b7368353a430dbffa1cf44f9b5949 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:00.377715 29556 tablet_service.cc:1467] Processing CreateTablet for tablet db9c5ce233da4b1a9ce70ae04aef7658 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8011864bab7b4506a89a8bbb019d244f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:00.377934 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db9c5ce233da4b1a9ce70ae04aef7658. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:00.378073 29426 tablet_service.cc:1467] Processing CreateTablet for tablet db9c5ce233da4b1a9ce70ae04aef7658 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8011864bab7b4506a89a8bbb019d244f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:00.378268 29426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db9c5ce233da4b1a9ce70ae04aef7658. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:00.378547 30263 tablet_bootstrap.cc:654] T 689b7368353a430dbffa1cf44f9b5949 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:00.379041 30265 tablet_bootstrap.cc:654] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:00.379500 30263 tablet_bootstrap.cc:492] T 689b7368353a430dbffa1cf44f9b5949 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:00.379573 30263 ts_tablet_manager.cc:1367] T 689b7368353a430dbffa1cf44f9b5949 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:00.379985 30263 raft_consensus.cc:348] T 689b7368353a430dbffa1cf44f9b5949 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:00.380088 30263 raft_consensus.cc:374] T 689b7368353a430dbffa1cf44f9b5949 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:00.380118 30263 raft_consensus.cc:729] T 689b7368353a430dbffa1cf44f9b5949 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:00.380234 30263 consensus_queue.cc:260] T 689b7368353a430dbffa1cf44f9b5949 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:00.380273 30265 tablet_bootstrap.cc:492] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:00.380229 29294 tablet_service.cc:1467] Processing CreateTablet for tablet 7fd046a0f0004e5fa6e0825cb626c2ff (DEFAULT_TABLE table=rebalance_test_table_1 [id=8011864bab7b4506a89a8bbb019d244f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:00.380352 30265 ts_tablet_manager.cc:1367] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:00.380442 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fd046a0f0004e5fa6e0825cb626c2ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:00.380472 30263 ts_tablet_manager.cc:1398] T 689b7368353a430dbffa1cf44f9b5949 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:00.380591 30263 tablet_bootstrap.cc:492] T 7fd046a0f0004e5fa6e0825cb626c2ff P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:00.380781 30265 raft_consensus.cc:348] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:00.380873 30265 raft_consensus.cc:374] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:00.380899 30265 raft_consensus.cc:729] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:00.381004 30265 consensus_queue.cc:260] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:00.381250 30265 ts_tablet_manager.cc:1398] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:00.381332 30263 tablet_bootstrap.cc:654] T 7fd046a0f0004e5fa6e0825cb626c2ff P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:00.381688 30261 tablet_bootstrap.cc:492] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:00.381757 30261 ts_tablet_manager.cc:1367] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 20:46:00.382162 30261 raft_consensus.cc:348] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:00.382264 30261 raft_consensus.cc:374] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:00.382289 30261 raft_consensus.cc:729] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:00.382833 30263 tablet_bootstrap.cc:492] T 7fd046a0f0004e5fa6e0825cb626c2ff P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:00.382903 30263 ts_tablet_manager.cc:1367] T 7fd046a0f0004e5fa6e0825cb626c2ff P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:00.382877 29295 tablet_service.cc:1467] Processing CreateTablet for tablet db9c5ce233da4b1a9ce70ae04aef7658 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8011864bab7b4506a89a8bbb019d244f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:00.383078 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db9c5ce233da4b1a9ce70ae04aef7658. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:00.383271 30263 raft_consensus.cc:348] T 7fd046a0f0004e5fa6e0825cb626c2ff P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:00.383355 30263 raft_consensus.cc:374] T 7fd046a0f0004e5fa6e0825cb626c2ff P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:00.383390 30263 raft_consensus.cc:729] T 7fd046a0f0004e5fa6e0825cb626c2ff P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:00.383502 30263 consensus_queue.cc:260] T 7fd046a0f0004e5fa6e0825cb626c2ff P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:00.383800 30263 ts_tablet_manager.cc:1398] T 7fd046a0f0004e5fa6e0825cb626c2ff P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:00.383946 30263 tablet_bootstrap.cc:492] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:00.382400 30261 consensus_queue.cc:260] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:00.384176 30261 ts_tablet_manager.cc:1398] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:00.384335 30261 tablet_bootstrap.cc:492] T 7fd046a0f0004e5fa6e0825cb626c2ff P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:00.384892 30263 tablet_bootstrap.cc:654] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:00.385327 30261 tablet_bootstrap.cc:654] T 7fd046a0f0004e5fa6e0825cb626c2ff P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:00.385483 30262 tablet_bootstrap.cc:492] T 689b7368353a430dbffa1cf44f9b5949 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:00.385537 30262 ts_tablet_manager.cc:1367] T 689b7368353a430dbffa1cf44f9b5949 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 20:46:00.385905 30263 tablet_bootstrap.cc:492] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:00.386023 30263 ts_tablet_manager.cc:1367] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 20:46:00.386164 30262 raft_consensus.cc:348] T 689b7368353a430dbffa1cf44f9b5949 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:00.386250 30262 raft_consensus.cc:374] T 689b7368353a430dbffa1cf44f9b5949 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:00.386278 30262 raft_consensus.cc:729] T 689b7368353a430dbffa1cf44f9b5949 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:00.386385 30262 consensus_queue.cc:260] T 689b7368353a430dbffa1cf44f9b5949 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:00.386421 30263 raft_consensus.cc:348] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:00.386541 30263 raft_consensus.cc:374] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:00.386572 30263 raft_consensus.cc:729] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:00.386653 30262 ts_tablet_manager.cc:1398] T 689b7368353a430dbffa1cf44f9b5949 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:00.386698 30263 consensus_queue.cc:260] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:00.386818 30262 tablet_bootstrap.cc:492] T db9c5ce233da4b1a9ce70ae04aef7658 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:00.386972 30263 ts_tablet_manager.cc:1398] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:00.386989 30261 tablet_bootstrap.cc:492] T 7fd046a0f0004e5fa6e0825cb626c2ff P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:00.387063 30261 ts_tablet_manager.cc:1367] T 7fd046a0f0004e5fa6e0825cb626c2ff P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:00.387416 30261 raft_consensus.cc:348] T 7fd046a0f0004e5fa6e0825cb626c2ff P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:00.387485 30261 raft_consensus.cc:374] T 7fd046a0f0004e5fa6e0825cb626c2ff P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:00.387475 30262 tablet_bootstrap.cc:654] T db9c5ce233da4b1a9ce70ae04aef7658 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:00.387503 30261 raft_consensus.cc:729] T 7fd046a0f0004e5fa6e0825cb626c2ff P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:00.387593 30261 consensus_queue.cc:260] T 7fd046a0f0004e5fa6e0825cb626c2ff P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:00.387825 30261 ts_tablet_manager.cc:1398] T 7fd046a0f0004e5fa6e0825cb626c2ff P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:00.387969 30261 tablet_bootstrap.cc:492] T db9c5ce233da4b1a9ce70ae04aef7658 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:00.388415 30262 tablet_bootstrap.cc:492] T db9c5ce233da4b1a9ce70ae04aef7658 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:00.388492 30262 ts_tablet_manager.cc:1367] T db9c5ce233da4b1a9ce70ae04aef7658 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:00.388881 30262 raft_consensus.cc:348] T db9c5ce233da4b1a9ce70ae04aef7658 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:00.388967 30262 raft_consensus.cc:374] T db9c5ce233da4b1a9ce70ae04aef7658 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:00.388998 30262 raft_consensus.cc:729] T db9c5ce233da4b1a9ce70ae04aef7658 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:00.389124 30262 consensus_queue.cc:260] T db9c5ce233da4b1a9ce70ae04aef7658 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:00.389437 30262 ts_tablet_manager.cc:1398] T db9c5ce233da4b1a9ce70ae04aef7658 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:00.390657 30261 tablet_bootstrap.cc:654] T db9c5ce233da4b1a9ce70ae04aef7658 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:00.391620 30261 tablet_bootstrap.cc:492] T db9c5ce233da4b1a9ce70ae04aef7658 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:00.391685 30261 ts_tablet_manager.cc:1367] T db9c5ce233da4b1a9ce70ae04aef7658 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:00.391999 30261 raft_consensus.cc:348] T db9c5ce233da4b1a9ce70ae04aef7658 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:00.392074 30261 raft_consensus.cc:374] T db9c5ce233da4b1a9ce70ae04aef7658 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:00.392098 30261 raft_consensus.cc:729] T db9c5ce233da4b1a9ce70ae04aef7658 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:00.392206 30261 consensus_queue.cc:260] T db9c5ce233da4b1a9ce70ae04aef7658 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:00.392684 30261 ts_tablet_manager.cc:1398] T db9c5ce233da4b1a9ce70ae04aef7658 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:00.408932 30240 raft_consensus.cc:482] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:00.409019 30240 raft_consensus.cc:504] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:00.409482 30240 leader_election.cc:288] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:00.409775 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db9c5ce233da4b1a9ce70ae04aef7658" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:00.409901 29577 raft_consensus.cc:2454] T db9c5ce233da4b1a9ce70ae04aef7658 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:00.410169 29380 leader_election.cc:302] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:00.410328 30240 raft_consensus.cc:2781] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:00.410382 30240 raft_consensus.cc:482] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:00.410418 30240 raft_consensus.cc:3037] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:00.411260 30240 raft_consensus.cc:504] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:00.411583 30240 leader_election.cc:288] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:00.411837 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db9c5ce233da4b1a9ce70ae04aef7658" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:00.411986 29577 raft_consensus.cc:3037] T db9c5ce233da4b1a9ce70ae04aef7658 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:00.412843 29577 raft_consensus.cc:2454] T db9c5ce233da4b1a9ce70ae04aef7658 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:00.413089 29380 leader_election.cc:302] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:00.413262 30240 raft_consensus.cc:2781] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:00.413368 30240 raft_consensus.cc:686] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:00.413368 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db9c5ce233da4b1a9ce70ae04aef7658" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:00.413486 29315 raft_consensus.cc:2454] T db9c5ce233da4b1a9ce70ae04aef7658 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:00.413661 30240 consensus_queue.cc:237] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:00.414631 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db9c5ce233da4b1a9ce70ae04aef7658" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:00.414744 29314 raft_consensus.cc:3037] T db9c5ce233da4b1a9ce70ae04aef7658 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:00.415352 29041 catalog_manager.cc:5484] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:00.415489 29314 raft_consensus.cc:2454] T db9c5ce233da4b1a9ce70ae04aef7658 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:00.570362 30178 raft_consensus.cc:482] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:00.570508 30178 raft_consensus.cc:504] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:00.570883 30178 leader_election.cc:288] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:00.571254 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "689b7368353a430dbffa1cf44f9b5949" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:00.571280 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "689b7368353a430dbffa1cf44f9b5949" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:00.571395 29577 raft_consensus.cc:2454] T 689b7368353a430dbffa1cf44f9b5949 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:00.571408 29445 raft_consensus.cc:2454] T 689b7368353a430dbffa1cf44f9b5949 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:00.571668 29249 leader_election.cc:302] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:00.571846 30178 raft_consensus.cc:2781] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:00.571907 30178 raft_consensus.cc:482] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:00.571933 30178 raft_consensus.cc:3037] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:00.572741 30178 raft_consensus.cc:504] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:00.573050 30178 leader_election.cc:288] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:00.573400 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "689b7368353a430dbffa1cf44f9b5949" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:00.573392 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "689b7368353a430dbffa1cf44f9b5949" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:00.573511 29577 raft_consensus.cc:3037] T 689b7368353a430dbffa1cf44f9b5949 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:00.573511 29445 raft_consensus.cc:3037] T 689b7368353a430dbffa1cf44f9b5949 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:00.574386 29577 raft_consensus.cc:2454] T 689b7368353a430dbffa1cf44f9b5949 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:00.574380 29445 raft_consensus.cc:2454] T 689b7368353a430dbffa1cf44f9b5949 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:00.574723 29250 leader_election.cc:302] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:00.574887 30178 raft_consensus.cc:2781] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:00.574980 30178 raft_consensus.cc:686] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:00.575112 30178 consensus_queue.cc:237] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:00.576632 29038 catalog_manager.cc:5484] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:00.691722 30187 raft_consensus.cc:482] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:00.691848 30187 raft_consensus.cc:504] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:00.692325 30187 leader_election.cc:288] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:00.692754 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fd046a0f0004e5fa6e0825cb626c2ff" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:00.692911 29445 raft_consensus.cc:2454] T 7fd046a0f0004e5fa6e0825cb626c2ff P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:00.693212 29119 leader_election.cc:302] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:00.693394 30187 raft_consensus.cc:2781] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:00.693465 30187 raft_consensus.cc:482] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:00.693506 30187 raft_consensus.cc:3037] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:00.694327 30187 raft_consensus.cc:504] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:00.694689 30187 leader_election.cc:288] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:00.695029 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fd046a0f0004e5fa6e0825cb626c2ff" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:00.695133 29445 raft_consensus.cc:3037] T 7fd046a0f0004e5fa6e0825cb626c2ff P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:00.695914 29445 raft_consensus.cc:2454] T 7fd046a0f0004e5fa6e0825cb626c2ff P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:00.696170 29119 leader_election.cc:302] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:00.696182 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fd046a0f0004e5fa6e0825cb626c2ff" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:00.696213 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fd046a0f0004e5fa6e0825cb626c2ff" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:00.696302 29314 raft_consensus.cc:2454] T 7fd046a0f0004e5fa6e0825cb626c2ff P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:00.696342 30187 raft_consensus.cc:2781] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:00.696419 30187 raft_consensus.cc:686] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:00.696552 30187 consensus_queue.cc:237] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:00.698098 29038 catalog_manager.cc:5484] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
W20240502 20:46:00.835863 30279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:00.836062 30279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:00.839071 30279 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20240502 20:46:00.917294 29577 raft_consensus.cc:1261] T 689b7368353a430dbffa1cf44f9b5949 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:00.918605 29315 raft_consensus.cc:3037] T 7fd046a0f0004e5fa6e0825cb626c2ff P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:00.918731 29445 raft_consensus.cc:1261] T 7fd046a0f0004e5fa6e0825cb626c2ff P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:00.919255 30195 consensus_queue.cc:1035] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:00.919682 29315 raft_consensus.cc:1261] T 7fd046a0f0004e5fa6e0825cb626c2ff P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:00.920122 29446 raft_consensus.cc:1261] T 689b7368353a430dbffa1cf44f9b5949 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:00.920094 30178 consensus_queue.cc:1035] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:00.920524 30178 consensus_queue.cc:1035] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:00.924099 30195 consensus_queue.cc:1035] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:00.942557 29314 raft_consensus.cc:1261] T db9c5ce233da4b1a9ce70ae04aef7658 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:00.942870 29577 raft_consensus.cc:1261] T db9c5ce233da4b1a9ce70ae04aef7658 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:00.943507 30314 consensus_queue.cc:1035] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:00.945374 30310 consensus_queue.cc:1035] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 99.647 ms
time per row: 0.0498235 ms
I20240502 20:46:00.978579 29037 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:00.983505 29162 tablet_service.cc:1467] Processing CreateTablet for tablet 1227d2743d824309949815354bcf7fce (DEFAULT_TABLE table=rebalance_test_table_2 [id=0c689bc5ef924b3eae28a1e2f63671df]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:00.983551 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 1227d2743d824309949815354bcf7fce (DEFAULT_TABLE table=rebalance_test_table_2 [id=0c689bc5ef924b3eae28a1e2f63671df]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:00.983806 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1227d2743d824309949815354bcf7fce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:00.983822 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1227d2743d824309949815354bcf7fce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:00.984380 29164 tablet_service.cc:1467] Processing CreateTablet for tablet e1459e78fecb466db1aa62cb7f53326a (DEFAULT_TABLE table=rebalance_test_table_2 [id=0c689bc5ef924b3eae28a1e2f63671df]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:00.984601 29164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1459e78fecb466db1aa62cb7f53326a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:00.985744 30336 tablet_bootstrap.cc:492] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:00.986061 29163 tablet_service.cc:1467] Processing CreateTablet for tablet 68eafb3578684972acc7ebdfe309a955 (DEFAULT_TABLE table=rebalance_test_table_2 [id=0c689bc5ef924b3eae28a1e2f63671df]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:00.986200 29556 tablet_service.cc:1467] Processing CreateTablet for tablet 1227d2743d824309949815354bcf7fce (DEFAULT_TABLE table=rebalance_test_table_2 [id=0c689bc5ef924b3eae28a1e2f63671df]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:00.986285 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68eafb3578684972acc7ebdfe309a955. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:00.986439 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1227d2743d824309949815354bcf7fce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:00.987123 30336 tablet_bootstrap.cc:654] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:00.988077 30337 tablet_bootstrap.cc:492] T 1227d2743d824309949815354bcf7fce P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:00.988390 30336 tablet_bootstrap.cc:492] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:00.988488 30339 tablet_bootstrap.cc:492] T 1227d2743d824309949815354bcf7fce P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:00.988549 30336 ts_tablet_manager.cc:1367] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:00.988830 30337 tablet_bootstrap.cc:654] T 1227d2743d824309949815354bcf7fce P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:00.989123 30336 raft_consensus.cc:348] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:00.989313 30336 raft_consensus.cc:374] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:00.989383 30336 raft_consensus.cc:729] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:00.989569 30336 consensus_queue.cc:260] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:00.989882 30336 ts_tablet_manager.cc:1398] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:00.990047 30336 tablet_bootstrap.cc:492] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:00.990077 30339 tablet_bootstrap.cc:654] T 1227d2743d824309949815354bcf7fce P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:00.990854 30336 tablet_bootstrap.cc:654] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:00.992130 30336 tablet_bootstrap.cc:492] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:00.992211 30336 ts_tablet_manager.cc:1367] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:00.992585 30336 raft_consensus.cc:348] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:00.992676 30336 raft_consensus.cc:374] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:00.992700 30336 raft_consensus.cc:729] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:00.992839 30336 consensus_queue.cc:260] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:00.992902 30339 tablet_bootstrap.cc:492] T 1227d2743d824309949815354bcf7fce P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:00.992977 30339 ts_tablet_manager.cc:1367] T 1227d2743d824309949815354bcf7fce P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 20:46:00.993136 30336 ts_tablet_manager.cc:1398] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:00.993263 30336 tablet_bootstrap.cc:492] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:00.993556 30339 raft_consensus.cc:348] T 1227d2743d824309949815354bcf7fce P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:00.993649 30339 raft_consensus.cc:374] T 1227d2743d824309949815354bcf7fce P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:00.993680 30339 raft_consensus.cc:729] T 1227d2743d824309949815354bcf7fce P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:00.993803 30339 consensus_queue.cc:260] T 1227d2743d824309949815354bcf7fce P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:00.993988 30336 tablet_bootstrap.cc:654] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:00.994046 30339 ts_tablet_manager.cc:1398] T 1227d2743d824309949815354bcf7fce P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:00.988617 29557 tablet_service.cc:1467] Processing CreateTablet for tablet 68eafb3578684972acc7ebdfe309a955 (DEFAULT_TABLE table=rebalance_test_table_2 [id=0c689bc5ef924b3eae28a1e2f63671df]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:00.994639 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68eafb3578684972acc7ebdfe309a955. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:00.994987 30336 tablet_bootstrap.cc:492] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:00.995050 30336 ts_tablet_manager.cc:1367] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:00.995554 30336 raft_consensus.cc:348] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:00.995636 30336 raft_consensus.cc:374] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:00.995664 30336 raft_consensus.cc:729] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:00.995781 30336 consensus_queue.cc:260] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:00.996040 30336 ts_tablet_manager.cc:1398] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:00.996413 30339 tablet_bootstrap.cc:492] T 68eafb3578684972acc7ebdfe309a955 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:00.988906 29555 tablet_service.cc:1467] Processing CreateTablet for tablet e1459e78fecb466db1aa62cb7f53326a (DEFAULT_TABLE table=rebalance_test_table_2 [id=0c689bc5ef924b3eae28a1e2f63671df]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:00.996677 29555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1459e78fecb466db1aa62cb7f53326a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:00.997107 30339 tablet_bootstrap.cc:654] T 68eafb3578684972acc7ebdfe309a955 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:00.998344 30339 tablet_bootstrap.cc:492] T 68eafb3578684972acc7ebdfe309a955 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:00.998420 30339 ts_tablet_manager.cc:1367] T 68eafb3578684972acc7ebdfe309a955 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:00.998487 29426 tablet_service.cc:1467] Processing CreateTablet for tablet 68eafb3578684972acc7ebdfe309a955 (DEFAULT_TABLE table=rebalance_test_table_2 [id=0c689bc5ef924b3eae28a1e2f63671df]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:00.998595 29295 tablet_service.cc:1467] Processing CreateTablet for tablet e1459e78fecb466db1aa62cb7f53326a (DEFAULT_TABLE table=rebalance_test_table_2 [id=0c689bc5ef924b3eae28a1e2f63671df]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:00.998724 29426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68eafb3578684972acc7ebdfe309a955. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:00.998811 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1459e78fecb466db1aa62cb7f53326a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:01.000103 30339 raft_consensus.cc:348] T 68eafb3578684972acc7ebdfe309a955 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:01.000219 30339 raft_consensus.cc:374] T 68eafb3578684972acc7ebdfe309a955 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:01.000265 30339 raft_consensus.cc:729] T 68eafb3578684972acc7ebdfe309a955 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:01.000422 30339 consensus_queue.cc:260] T 68eafb3578684972acc7ebdfe309a955 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:01.000753 30339 ts_tablet_manager.cc:1398] T 68eafb3578684972acc7ebdfe309a955 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:01.000955 30339 tablet_bootstrap.cc:492] T e1459e78fecb466db1aa62cb7f53326a P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:01.001161 30345 tablet_bootstrap.cc:492] T 68eafb3578684972acc7ebdfe309a955 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:01.001809 30339 tablet_bootstrap.cc:654] T e1459e78fecb466db1aa62cb7f53326a P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:01.001999 30345 tablet_bootstrap.cc:654] T 68eafb3578684972acc7ebdfe309a955 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:01.002825 30339 tablet_bootstrap.cc:492] T e1459e78fecb466db1aa62cb7f53326a P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:01.002946 30339 ts_tablet_manager.cc:1367] T e1459e78fecb466db1aa62cb7f53326a P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 20:46:01.003216 30345 tablet_bootstrap.cc:492] T 68eafb3578684972acc7ebdfe309a955 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:01.003253 30195 raft_consensus.cc:482] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:01.003294 30345 ts_tablet_manager.cc:1367] T 68eafb3578684972acc7ebdfe309a955 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:01.003351 30195 raft_consensus.cc:504] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:01.003589 30339 raft_consensus.cc:348] T e1459e78fecb466db1aa62cb7f53326a P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:01.003696 30339 raft_consensus.cc:374] T e1459e78fecb466db1aa62cb7f53326a P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:01.003664 30345 raft_consensus.cc:348] T 68eafb3578684972acc7ebdfe309a955 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:01.003706 30195 leader_election.cc:288] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:01.003741 30345 raft_consensus.cc:374] T 68eafb3578684972acc7ebdfe309a955 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:01.003780 30345 raft_consensus.cc:729] T 68eafb3578684972acc7ebdfe309a955 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:01.003756 30339 raft_consensus.cc:729] T e1459e78fecb466db1aa62cb7f53326a P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:01.003885 30345 consensus_queue.cc:260] T 68eafb3578684972acc7ebdfe309a955 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:01.003902 30339 consensus_queue.cc:260] T e1459e78fecb466db1aa62cb7f53326a P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:01.004114 30345 ts_tablet_manager.cc:1398] T 68eafb3578684972acc7ebdfe309a955 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:01.004089 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1227d2743d824309949815354bcf7fce" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:01.004182 29577 raft_consensus.cc:2454] T 1227d2743d824309949815354bcf7fce P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:01.004328 30339 ts_tablet_manager.cc:1398] T e1459e78fecb466db1aa62cb7f53326a P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:01.004953 29118 leader_election.cc:302] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:01.005182 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1227d2743d824309949815354bcf7fce" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
W20240502 20:46:01.005666 29119 leader_election.cc:341] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): Illegal state: must be running to vote when last-logged opid is not known
I20240502 20:46:01.006078 30195 raft_consensus.cc:2781] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:01.006142 30195 raft_consensus.cc:482] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:01.006186 30195 raft_consensus.cc:3037] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:01.006922 30195 raft_consensus.cc:504] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:01.007213 30195 leader_election.cc:288] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:01.007452 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1227d2743d824309949815354bcf7fce" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:01.007655 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1227d2743d824309949815354bcf7fce" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
W20240502 20:46:01.007715 29119 leader_election.cc:341] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): Illegal state: must be running to vote when last-logged opid is not known
I20240502 20:46:01.007755 29577 raft_consensus.cc:3037] T 1227d2743d824309949815354bcf7fce P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:01.008407 29577 raft_consensus.cc:2454] T 1227d2743d824309949815354bcf7fce P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:01.008649 29118 leader_election.cc:302] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057; no voters: 09258ac095664fa99caf0707262cde13
I20240502 20:46:01.008827 30195 raft_consensus.cc:2781] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:01.008910 30195 raft_consensus.cc:686] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:01.009030 30195 consensus_queue.cc:237] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:01.010481 29041 catalog_manager.cc:5484] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:01.014978 30337 tablet_bootstrap.cc:492] T 1227d2743d824309949815354bcf7fce P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:01.015059 30337 ts_tablet_manager.cc:1367] T 1227d2743d824309949815354bcf7fce P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.027s user 0.001s sys 0.000s
I20240502 20:46:01.015421 30337 raft_consensus.cc:348] T 1227d2743d824309949815354bcf7fce P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:01.015506 30337 raft_consensus.cc:374] T 1227d2743d824309949815354bcf7fce P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:01.015533 30337 raft_consensus.cc:729] T 1227d2743d824309949815354bcf7fce P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:01.015648 30337 consensus_queue.cc:260] T 1227d2743d824309949815354bcf7fce P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:01.015861 30337 ts_tablet_manager.cc:1398] T 1227d2743d824309949815354bcf7fce P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:01.016009 30337 tablet_bootstrap.cc:492] T e1459e78fecb466db1aa62cb7f53326a P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:01.016656 30337 tablet_bootstrap.cc:654] T e1459e78fecb466db1aa62cb7f53326a P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:01.017588 30337 tablet_bootstrap.cc:492] T e1459e78fecb466db1aa62cb7f53326a P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:01.017657 30337 ts_tablet_manager.cc:1367] T e1459e78fecb466db1aa62cb7f53326a P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:01.018003 30337 raft_consensus.cc:348] T e1459e78fecb466db1aa62cb7f53326a P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:01.018078 30337 raft_consensus.cc:374] T e1459e78fecb466db1aa62cb7f53326a P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:01.018103 30337 raft_consensus.cc:729] T e1459e78fecb466db1aa62cb7f53326a P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:01.018195 30337 consensus_queue.cc:260] T e1459e78fecb466db1aa62cb7f53326a P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:01.018415 30337 ts_tablet_manager.cc:1398] T e1459e78fecb466db1aa62cb7f53326a P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:01.080907 30187 raft_consensus.cc:482] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:01.081053 30187 raft_consensus.cc:504] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:01.081420 30187 leader_election.cc:288] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:01.081785 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eafb3578684972acc7ebdfe309a955" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:01.081823 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eafb3578684972acc7ebdfe309a955" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:01.081938 29577 raft_consensus.cc:2454] T 68eafb3578684972acc7ebdfe309a955 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:01.081936 29446 raft_consensus.cc:2454] T 68eafb3578684972acc7ebdfe309a955 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:01.082232 29118 leader_election.cc:302] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:01.082417 30187 raft_consensus.cc:2781] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:01.082502 30187 raft_consensus.cc:482] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:01.082530 30187 raft_consensus.cc:3037] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:01.083287 30187 raft_consensus.cc:504] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:01.083595 30187 leader_election.cc:288] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:01.083906 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eafb3578684972acc7ebdfe309a955" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:01.083914 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68eafb3578684972acc7ebdfe309a955" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:01.084022 29577 raft_consensus.cc:3037] T 68eafb3578684972acc7ebdfe309a955 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:01.084022 29446 raft_consensus.cc:3037] T 68eafb3578684972acc7ebdfe309a955 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:01.084882 29446 raft_consensus.cc:2454] T 68eafb3578684972acc7ebdfe309a955 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:01.084882 29577 raft_consensus.cc:2454] T 68eafb3578684972acc7ebdfe309a955 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:01.085140 29119 leader_election.cc:302] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:01.085295 30187 raft_consensus.cc:2781] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:01.085387 30187 raft_consensus.cc:686] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:01.085512 30187 consensus_queue.cc:237] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:01.086982 29041 catalog_manager.cc:5484] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:01.150300 30173 raft_consensus.cc:482] T 1227d2743d824309949815354bcf7fce P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:01.150417 30173 raft_consensus.cc:504] T 1227d2743d824309949815354bcf7fce P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:01.150837 30173 leader_election.cc:288] T 1227d2743d824309949815354bcf7fce P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:01.151196 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1227d2743d824309949815354bcf7fce" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:01.151221 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1227d2743d824309949815354bcf7fce" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:01.151397 29577 raft_consensus.cc:2379] T 1227d2743d824309949815354bcf7fce P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 09258ac095664fa99caf0707262cde13 in current term 1: Already voted for candidate 7796ea3fba294d17a34386873969431d in this term.
I20240502 20:46:01.151698 29249 leader_election.cc:302] T 1227d2743d824309949815354bcf7fce P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 09258ac095664fa99caf0707262cde13; no voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057
I20240502 20:46:01.151860 30173 raft_consensus.cc:3037] T 1227d2743d824309949815354bcf7fce P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:01.152663 30173 raft_consensus.cc:2726] T 1227d2743d824309949815354bcf7fce P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 20:46:01.389479 30187 raft_consensus.cc:482] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:01.389612 30187 raft_consensus.cc:504] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:01.389969 30187 leader_election.cc:288] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:01.390347 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1459e78fecb466db1aa62cb7f53326a" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:01.390372 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1459e78fecb466db1aa62cb7f53326a" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:01.390496 29577 raft_consensus.cc:2454] T e1459e78fecb466db1aa62cb7f53326a P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:01.390532 29315 raft_consensus.cc:2454] T e1459e78fecb466db1aa62cb7f53326a P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:01.390774 29118 leader_election.cc:302] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:01.390967 30187 raft_consensus.cc:2781] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:01.391028 30187 raft_consensus.cc:482] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:01.391053 30187 raft_consensus.cc:3037] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:01.391798 30187 raft_consensus.cc:504] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:01.392102 30187 leader_election.cc:288] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:01.392402 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1459e78fecb466db1aa62cb7f53326a" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:01.392421 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1459e78fecb466db1aa62cb7f53326a" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:01.392527 29577 raft_consensus.cc:3037] T e1459e78fecb466db1aa62cb7f53326a P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:01.392527 29315 raft_consensus.cc:3037] T e1459e78fecb466db1aa62cb7f53326a P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:01.393383 29315 raft_consensus.cc:2454] T e1459e78fecb466db1aa62cb7f53326a P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:01.393383 29577 raft_consensus.cc:2454] T e1459e78fecb466db1aa62cb7f53326a P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:01.393651 29119 leader_election.cc:302] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:01.393829 30187 raft_consensus.cc:2781] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:01.393944 30187 raft_consensus.cc:686] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:01.394083 30187 consensus_queue.cc:237] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:01.395562 29041 catalog_manager.cc:5484] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:01.400188 30187 consensus_queue.cc:1035] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:01.402740 30187 consensus_queue.cc:1035] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:01.499464 30187 consensus_queue.cc:1035] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:01.502601 30195 consensus_queue.cc:1035] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 20:46:01.537335 30354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:01.537533 30354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:01.540437 30354 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20240502 20:46:01.608780 29315 raft_consensus.cc:1261] T e1459e78fecb466db1aa62cb7f53326a P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:01.609412 29577 raft_consensus.cc:1261] T e1459e78fecb466db1aa62cb7f53326a P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:01.614321 30195 consensus_queue.cc:1035] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:01.615144 30195 consensus_queue.cc:1035] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 83.6458 ms
time per row: 0.0418229 ms
I20240502 20:46:01.663729 29037 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:01.668730 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 4fd651e80e004dc58d7769f227af58a8 (DEFAULT_TABLE table=rebalance_test_table_3 [id=aec26ee01a7a4b05887071d6b1783fb0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:01.669032 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fd651e80e004dc58d7769f227af58a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:01.669566 29164 tablet_service.cc:1467] Processing CreateTablet for tablet 2d72ef6e7e8d473baaff8f527c02332b (DEFAULT_TABLE table=rebalance_test_table_3 [id=aec26ee01a7a4b05887071d6b1783fb0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:01.669857 29164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d72ef6e7e8d473baaff8f527c02332b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:01.669847 29557 tablet_service.cc:1467] Processing CreateTablet for tablet a2bf196bcab440c48aa4ea893997573b (DEFAULT_TABLE table=rebalance_test_table_3 [id=aec26ee01a7a4b05887071d6b1783fb0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:01.670095 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2bf196bcab440c48aa4ea893997573b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:01.669852 29555 tablet_service.cc:1467] Processing CreateTablet for tablet 2d72ef6e7e8d473baaff8f527c02332b (DEFAULT_TABLE table=rebalance_test_table_3 [id=aec26ee01a7a4b05887071d6b1783fb0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:01.670742 29555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d72ef6e7e8d473baaff8f527c02332b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:01.669566 29163 tablet_service.cc:1467] Processing CreateTablet for tablet 4fd651e80e004dc58d7769f227af58a8 (DEFAULT_TABLE table=rebalance_test_table_3 [id=aec26ee01a7a4b05887071d6b1783fb0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:01.672253 29426 tablet_service.cc:1467] Processing CreateTablet for tablet 4fd651e80e004dc58d7769f227af58a8 (DEFAULT_TABLE table=rebalance_test_table_3 [id=aec26ee01a7a4b05887071d6b1783fb0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:01.672433 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fd651e80e004dc58d7769f227af58a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:01.672544 29426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fd651e80e004dc58d7769f227af58a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:01.672736 29295 tablet_service.cc:1467] Processing CreateTablet for tablet a2bf196bcab440c48aa4ea893997573b (DEFAULT_TABLE table=rebalance_test_table_3 [id=aec26ee01a7a4b05887071d6b1783fb0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:01.672974 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2bf196bcab440c48aa4ea893997573b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:01.673303 30406 tablet_bootstrap.cc:492] T 4fd651e80e004dc58d7769f227af58a8 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:01.674201 30406 tablet_bootstrap.cc:654] T 4fd651e80e004dc58d7769f227af58a8 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:01.674700 29425 tablet_service.cc:1467] Processing CreateTablet for tablet 2d72ef6e7e8d473baaff8f527c02332b (DEFAULT_TABLE table=rebalance_test_table_3 [id=aec26ee01a7a4b05887071d6b1783fb0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:01.674922 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d72ef6e7e8d473baaff8f527c02332b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:01.675086 30408 tablet_bootstrap.cc:492] T a2bf196bcab440c48aa4ea893997573b P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:01.675911 30408 tablet_bootstrap.cc:654] T a2bf196bcab440c48aa4ea893997573b P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:01.676853 29424 tablet_service.cc:1467] Processing CreateTablet for tablet a2bf196bcab440c48aa4ea893997573b (DEFAULT_TABLE table=rebalance_test_table_3 [id=aec26ee01a7a4b05887071d6b1783fb0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:01.677027 30409 tablet_bootstrap.cc:492] T 4fd651e80e004dc58d7769f227af58a8 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:01.677083 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2bf196bcab440c48aa4ea893997573b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:01.677186 30406 tablet_bootstrap.cc:492] T 4fd651e80e004dc58d7769f227af58a8 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:01.677217 30408 tablet_bootstrap.cc:492] T a2bf196bcab440c48aa4ea893997573b P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:01.677248 30407 tablet_bootstrap.cc:492] T 2d72ef6e7e8d473baaff8f527c02332b P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:01.677331 30408 ts_tablet_manager.cc:1367] T a2bf196bcab440c48aa4ea893997573b P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:01.677470 30406 ts_tablet_manager.cc:1367] T 4fd651e80e004dc58d7769f227af58a8 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 20:46:01.678000 30407 tablet_bootstrap.cc:654] T 2d72ef6e7e8d473baaff8f527c02332b P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:01.678210 30406 raft_consensus.cc:348] T 4fd651e80e004dc58d7769f227af58a8 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:01.678364 30406 raft_consensus.cc:374] T 4fd651e80e004dc58d7769f227af58a8 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:01.678436 30406 raft_consensus.cc:729] T 4fd651e80e004dc58d7769f227af58a8 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:01.678647 30406 consensus_queue.cc:260] T 4fd651e80e004dc58d7769f227af58a8 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:01.678664 30408 raft_consensus.cc:348] T a2bf196bcab440c48aa4ea893997573b P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:01.678754 30408 raft_consensus.cc:374] T a2bf196bcab440c48aa4ea893997573b P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:01.678848 30408 raft_consensus.cc:729] T a2bf196bcab440c48aa4ea893997573b P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:01.678997 30408 consensus_queue.cc:260] T a2bf196bcab440c48aa4ea893997573b P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:01.679172 30406 ts_tablet_manager.cc:1398] T 4fd651e80e004dc58d7769f227af58a8 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:01.679366 30406 tablet_bootstrap.cc:492] T a2bf196bcab440c48aa4ea893997573b P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:01.679412 30409 tablet_bootstrap.cc:654] T 4fd651e80e004dc58d7769f227af58a8 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:01.679461 30408 ts_tablet_manager.cc:1398] T a2bf196bcab440c48aa4ea893997573b P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:01.679687 30408 tablet_bootstrap.cc:492] T 2d72ef6e7e8d473baaff8f527c02332b P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:01.680352 30407 tablet_bootstrap.cc:492] T 2d72ef6e7e8d473baaff8f527c02332b P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:01.680437 30406 tablet_bootstrap.cc:654] T a2bf196bcab440c48aa4ea893997573b P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:01.680445 30407 ts_tablet_manager.cc:1367] T 2d72ef6e7e8d473baaff8f527c02332b P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 20:46:01.680861 30407 raft_consensus.cc:348] T 2d72ef6e7e8d473baaff8f527c02332b P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:01.680968 30407 raft_consensus.cc:374] T 2d72ef6e7e8d473baaff8f527c02332b P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:01.681013 30407 raft_consensus.cc:729] T 2d72ef6e7e8d473baaff8f527c02332b P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:01.681157 30407 consensus_queue.cc:260] T 2d72ef6e7e8d473baaff8f527c02332b P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:01.681510 30408 tablet_bootstrap.cc:654] T 2d72ef6e7e8d473baaff8f527c02332b P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:01.681505 30407 ts_tablet_manager.cc:1398] T 2d72ef6e7e8d473baaff8f527c02332b P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:01.681700 30407 tablet_bootstrap.cc:492] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:01.682593 30407 tablet_bootstrap.cc:654] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:01.683724 30409 tablet_bootstrap.cc:492] T 4fd651e80e004dc58d7769f227af58a8 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:01.683815 30409 ts_tablet_manager.cc:1367] T 4fd651e80e004dc58d7769f227af58a8 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 20:46:01.684227 30409 raft_consensus.cc:348] T 4fd651e80e004dc58d7769f227af58a8 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:01.684329 30409 raft_consensus.cc:374] T 4fd651e80e004dc58d7769f227af58a8 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:01.684365 30409 raft_consensus.cc:729] T 4fd651e80e004dc58d7769f227af58a8 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:01.684486 30409 consensus_queue.cc:260] T 4fd651e80e004dc58d7769f227af58a8 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:01.684710 30408 tablet_bootstrap.cc:492] T 2d72ef6e7e8d473baaff8f527c02332b P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:01.684762 30409 ts_tablet_manager.cc:1398] T 4fd651e80e004dc58d7769f227af58a8 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:01.684787 30408 ts_tablet_manager.cc:1367] T 2d72ef6e7e8d473baaff8f527c02332b P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 20:46:01.684935 30409 tablet_bootstrap.cc:492] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:01.685158 30408 raft_consensus.cc:348] T 2d72ef6e7e8d473baaff8f527c02332b P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:01.685256 30408 raft_consensus.cc:374] T 2d72ef6e7e8d473baaff8f527c02332b P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:01.685284 30408 raft_consensus.cc:729] T 2d72ef6e7e8d473baaff8f527c02332b P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:01.685402 30408 consensus_queue.cc:260] T 2d72ef6e7e8d473baaff8f527c02332b P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:01.685636 30409 tablet_bootstrap.cc:654] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:01.685662 30408 ts_tablet_manager.cc:1398] T 2d72ef6e7e8d473baaff8f527c02332b P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:01.686372 30407 tablet_bootstrap.cc:492] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:01.686450 30407 ts_tablet_manager.cc:1367] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 20:46:01.686856 30409 tablet_bootstrap.cc:492] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:01.686856 30407 raft_consensus.cc:348] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:01.686957 30407 raft_consensus.cc:374] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:01.686995 30407 raft_consensus.cc:729] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:01.687106 30407 consensus_queue.cc:260] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:01.687439 30407 ts_tablet_manager.cc:1398] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:01.687711 30409 ts_tablet_manager.cc:1367] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:01.688066 30406 tablet_bootstrap.cc:492] T a2bf196bcab440c48aa4ea893997573b P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:01.688161 30406 ts_tablet_manager.cc:1367] T a2bf196bcab440c48aa4ea893997573b P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20240502 20:46:01.688593 30406 raft_consensus.cc:348] T a2bf196bcab440c48aa4ea893997573b P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:01.688699 30406 raft_consensus.cc:374] T a2bf196bcab440c48aa4ea893997573b P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:01.688728 30406 raft_consensus.cc:729] T a2bf196bcab440c48aa4ea893997573b P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:01.688860 30406 consensus_queue.cc:260] T a2bf196bcab440c48aa4ea893997573b P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:01.689100 30406 ts_tablet_manager.cc:1398] T a2bf196bcab440c48aa4ea893997573b P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 20:46:01.688975 30409 raft_consensus.cc:348] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:01.689406 30409 raft_consensus.cc:374] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:01.689458 30409 raft_consensus.cc:729] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:01.689576 30409 consensus_queue.cc:260] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:01.689850 30409 ts_tablet_manager.cc:1398] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240502 20:46:01.689996 30409 tablet_bootstrap.cc:492] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:01.690825 30409 tablet_bootstrap.cc:654] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:01.691731 30409 tablet_bootstrap.cc:492] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:01.691807 30409 ts_tablet_manager.cc:1367] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:01.692112 30409 raft_consensus.cc:348] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:01.692193 30409 raft_consensus.cc:374] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:01.692234 30409 raft_consensus.cc:729] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:01.692345 30409 consensus_queue.cc:260] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:01.692584 30409 ts_tablet_manager.cc:1398] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:01.694828 30335 raft_consensus.cc:482] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:01.694922 30335 raft_consensus.cc:504] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:01.695261 30335 leader_election.cc:288] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:01.695526 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d72ef6e7e8d473baaff8f527c02332b" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:01.695597 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d72ef6e7e8d473baaff8f527c02332b" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:01.695647 29175 raft_consensus.cc:2454] T 2d72ef6e7e8d473baaff8f527c02332b P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:01.695719 29575 raft_consensus.cc:2454] T 2d72ef6e7e8d473baaff8f527c02332b P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:01.695957 29380 leader_election.cc:302] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:01.696101 30335 raft_consensus.cc:2781] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:01.696174 30335 raft_consensus.cc:482] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:01.696218 30335 raft_consensus.cc:3037] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:01.696930 30335 raft_consensus.cc:504] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:01.697324 30335 leader_election.cc:288] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:01.697603 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d72ef6e7e8d473baaff8f527c02332b" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:01.697659 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d72ef6e7e8d473baaff8f527c02332b" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:01.697707 29575 raft_consensus.cc:3037] T 2d72ef6e7e8d473baaff8f527c02332b P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:01.697738 29175 raft_consensus.cc:3037] T 2d72ef6e7e8d473baaff8f527c02332b P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:01.698576 29175 raft_consensus.cc:2454] T 2d72ef6e7e8d473baaff8f527c02332b P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:01.698576 29575 raft_consensus.cc:2454] T 2d72ef6e7e8d473baaff8f527c02332b P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:01.698839 29382 leader_election.cc:302] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:01.698997 30335 raft_consensus.cc:2781] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:01.699193 30335 raft_consensus.cc:686] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:01.699324 30335 consensus_queue.cc:237] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:01.700671 29033 catalog_manager.cc:5484] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:01.705106 30187 raft_consensus.cc:482] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:01.705178 30187 raft_consensus.cc:504] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:01.705463 30187 leader_election.cc:288] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:01.705806 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fd651e80e004dc58d7769f227af58a8" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:01.705832 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fd651e80e004dc58d7769f227af58a8" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:01.705943 29315 raft_consensus.cc:2454] T 4fd651e80e004dc58d7769f227af58a8 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:01.705942 29446 raft_consensus.cc:2454] T 4fd651e80e004dc58d7769f227af58a8 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:01.706192 29119 leader_election.cc:302] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:01.706352 30187 raft_consensus.cc:2781] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:01.706413 30187 raft_consensus.cc:482] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:01.706482 30187 raft_consensus.cc:3037] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:01.707152 30187 raft_consensus.cc:504] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:01.707468 30187 leader_election.cc:288] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:01.707716 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fd651e80e004dc58d7769f227af58a8" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:01.707816 29315 raft_consensus.cc:3037] T 4fd651e80e004dc58d7769f227af58a8 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:01.707875 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fd651e80e004dc58d7769f227af58a8" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:01.707969 29446 raft_consensus.cc:3037] T 4fd651e80e004dc58d7769f227af58a8 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:01.708694 29446 raft_consensus.cc:2454] T 4fd651e80e004dc58d7769f227af58a8 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:01.708694 29315 raft_consensus.cc:2454] T 4fd651e80e004dc58d7769f227af58a8 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:01.708969 29119 leader_election.cc:302] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:01.709115 30187 raft_consensus.cc:2781] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:01.709204 30187 raft_consensus.cc:686] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:01.709321 30187 consensus_queue.cc:237] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:01.710690 29033 catalog_manager.cc:5484] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:01.809180 30335 raft_consensus.cc:482] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:01.809300 30335 raft_consensus.cc:504] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:01.809697 30335 leader_election.cc:288] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:01.810079 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2bf196bcab440c48aa4ea893997573b" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:01.810081 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2bf196bcab440c48aa4ea893997573b" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:01.810209 29315 raft_consensus.cc:2454] T a2bf196bcab440c48aa4ea893997573b P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:01.810216 29575 raft_consensus.cc:2454] T a2bf196bcab440c48aa4ea893997573b P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:01.810524 29380 leader_election.cc:302] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:01.810690 30335 raft_consensus.cc:2781] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:01.810750 30335 raft_consensus.cc:482] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:01.810775 30335 raft_consensus.cc:3037] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:01.811530 30335 raft_consensus.cc:504] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:01.811851 30335 leader_election.cc:288] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:01.812176 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2bf196bcab440c48aa4ea893997573b" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:01.812179 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2bf196bcab440c48aa4ea893997573b" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:01.812263 29575 raft_consensus.cc:3037] T a2bf196bcab440c48aa4ea893997573b P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:01.812279 29315 raft_consensus.cc:3037] T a2bf196bcab440c48aa4ea893997573b P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:01.812945 29575 raft_consensus.cc:2454] T a2bf196bcab440c48aa4ea893997573b P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:01.813092 29315 raft_consensus.cc:2454] T a2bf196bcab440c48aa4ea893997573b P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:01.813206 29380 leader_election.cc:302] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:01.813372 30335 raft_consensus.cc:2781] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:01.813441 30335 raft_consensus.cc:686] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:01.813568 30335 consensus_queue.cc:237] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:01.815168 29033 catalog_manager.cc:5484] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
W20240502 20:46:01.924520 30423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:01.924713 30423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:01.927687 30423 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
I20240502 20:46:02.005411 29315 raft_consensus.cc:1261] T 4fd651e80e004dc58d7769f227af58a8 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:02.005913 29446 raft_consensus.cc:1261] T 4fd651e80e004dc58d7769f227af58a8 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:02.006430 30195 consensus_queue.cc:1035] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:02.006760 30195 consensus_queue.cc:1035] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:02.017526 29575 raft_consensus.cc:1261] T 2d72ef6e7e8d473baaff8f527c02332b P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:02.018214 30419 consensus_queue.cc:1035] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:02.018317 29575 raft_consensus.cc:1261] T a2bf196bcab440c48aa4ea893997573b P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:02.018960 30335 consensus_queue.cc:1035] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:02.019081 29175 raft_consensus.cc:1261] T 2d72ef6e7e8d473baaff8f527c02332b P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:02.023159 29314 raft_consensus.cc:1261] T a2bf196bcab440c48aa4ea893997573b P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:02.023923 30335 consensus_queue.cc:1035] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:02.026180 30335 consensus_queue.cc:1035] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 107.062 ms
time per row: 0.0535309 ms
I20240502 20:46:02.077720 29038 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:02.082633 29424 tablet_service.cc:1467] Processing CreateTablet for tablet 44010b796bae4b97ab7f17ee5cdffc30 (DEFAULT_TABLE table=rebalance_test_table_4 [id=b684c3e68fbe4841bf5ca5674beb20f1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:02.082700 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 44010b796bae4b97ab7f17ee5cdffc30 (DEFAULT_TABLE table=rebalance_test_table_4 [id=b684c3e68fbe4841bf5ca5674beb20f1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:02.082926 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44010b796bae4b97ab7f17ee5cdffc30. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:02.082924 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44010b796bae4b97ab7f17ee5cdffc30. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:02.084914 30406 tablet_bootstrap.cc:492] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:02.084914 30409 tablet_bootstrap.cc:492] T 44010b796bae4b97ab7f17ee5cdffc30 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:02.085156 29163 tablet_service.cc:1467] Processing CreateTablet for tablet 44010b796bae4b97ab7f17ee5cdffc30 (DEFAULT_TABLE table=rebalance_test_table_4 [id=b684c3e68fbe4841bf5ca5674beb20f1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:02.085408 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44010b796bae4b97ab7f17ee5cdffc30. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:02.085362 29164 tablet_service.cc:1467] Processing CreateTablet for tablet 6a3c10a388d8469bb04a21b48b4aafaa (DEFAULT_TABLE table=rebalance_test_table_4 [id=b684c3e68fbe4841bf5ca5674beb20f1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:02.085565 29164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a3c10a388d8469bb04a21b48b4aafaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:02.085841 30406 tablet_bootstrap.cc:654] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:02.085916 30409 tablet_bootstrap.cc:654] T 44010b796bae4b97ab7f17ee5cdffc30 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:02.087041 30409 tablet_bootstrap.cc:492] T 44010b796bae4b97ab7f17ee5cdffc30 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:02.087121 30409 ts_tablet_manager.cc:1367] T 44010b796bae4b97ab7f17ee5cdffc30 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:02.087538 30407 tablet_bootstrap.cc:492] T 6a3c10a388d8469bb04a21b48b4aafaa P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:02.087507 30409 raft_consensus.cc:348] T 44010b796bae4b97ab7f17ee5cdffc30 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:02.087605 30409 raft_consensus.cc:374] T 44010b796bae4b97ab7f17ee5cdffc30 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:02.087635 30409 raft_consensus.cc:729] T 44010b796bae4b97ab7f17ee5cdffc30 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:02.087755 30409 consensus_queue.cc:260] T 44010b796bae4b97ab7f17ee5cdffc30 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:02.087853 30406 tablet_bootstrap.cc:492] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:02.087925 30406 ts_tablet_manager.cc:1367] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 20:46:02.088037 30409 ts_tablet_manager.cc:1398] T 44010b796bae4b97ab7f17ee5cdffc30 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:02.088217 30407 tablet_bootstrap.cc:654] T 6a3c10a388d8469bb04a21b48b4aafaa P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:02.088315 30406 raft_consensus.cc:348] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:02.088397 30406 raft_consensus.cc:374] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:02.088423 30406 raft_consensus.cc:729] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:02.088538 30406 consensus_queue.cc:260] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:02.088783 30406 ts_tablet_manager.cc:1398] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:02.089916 30407 tablet_bootstrap.cc:492] T 6a3c10a388d8469bb04a21b48b4aafaa P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:02.089985 30407 ts_tablet_manager.cc:1367] T 6a3c10a388d8469bb04a21b48b4aafaa P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:02.090317 30407 raft_consensus.cc:348] T 6a3c10a388d8469bb04a21b48b4aafaa P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:02.090422 30407 raft_consensus.cc:374] T 6a3c10a388d8469bb04a21b48b4aafaa P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:02.090453 30407 raft_consensus.cc:729] T 6a3c10a388d8469bb04a21b48b4aafaa P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:02.090860 30407 consensus_queue.cc:260] T 6a3c10a388d8469bb04a21b48b4aafaa P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:02.091107 30407 ts_tablet_manager.cc:1398] T 6a3c10a388d8469bb04a21b48b4aafaa P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:02.091125 29555 tablet_service.cc:1467] Processing CreateTablet for tablet 6a3c10a388d8469bb04a21b48b4aafaa (DEFAULT_TABLE table=rebalance_test_table_4 [id=b684c3e68fbe4841bf5ca5674beb20f1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:02.091265 30407 tablet_bootstrap.cc:492] T 44010b796bae4b97ab7f17ee5cdffc30 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:02.091392 29555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a3c10a388d8469bb04a21b48b4aafaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:02.092000 30407 tablet_bootstrap.cc:654] T 44010b796bae4b97ab7f17ee5cdffc30 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:02.093462 30408 tablet_bootstrap.cc:492] T 6a3c10a388d8469bb04a21b48b4aafaa P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:02.094048 30407 tablet_bootstrap.cc:492] T 44010b796bae4b97ab7f17ee5cdffc30 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:02.094116 30407 ts_tablet_manager.cc:1367] T 44010b796bae4b97ab7f17ee5cdffc30 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:02.094127 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 6a3c10a388d8469bb04a21b48b4aafaa (DEFAULT_TABLE table=rebalance_test_table_4 [id=b684c3e68fbe4841bf5ca5674beb20f1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:02.094249 30408 tablet_bootstrap.cc:654] T 6a3c10a388d8469bb04a21b48b4aafaa P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:02.094344 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a3c10a388d8469bb04a21b48b4aafaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:02.094537 30407 raft_consensus.cc:348] T 44010b796bae4b97ab7f17ee5cdffc30 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:02.094626 30407 raft_consensus.cc:374] T 44010b796bae4b97ab7f17ee5cdffc30 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:02.094656 30407 raft_consensus.cc:729] T 44010b796bae4b97ab7f17ee5cdffc30 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:02.094781 30407 consensus_queue.cc:260] T 44010b796bae4b97ab7f17ee5cdffc30 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:02.095331 30407 ts_tablet_manager.cc:1398] T 44010b796bae4b97ab7f17ee5cdffc30 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:02.095444 29424 tablet_service.cc:1467] Processing CreateTablet for tablet fbd8fbbb18334242834ec7fa39184204 (DEFAULT_TABLE table=rebalance_test_table_4 [id=b684c3e68fbe4841bf5ca5674beb20f1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:02.095674 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbd8fbbb18334242834ec7fa39184204. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:02.096024 30406 tablet_bootstrap.cc:492] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:02.096812 30408 tablet_bootstrap.cc:492] T 6a3c10a388d8469bb04a21b48b4aafaa P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:02.096858 30406 tablet_bootstrap.cc:654] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:02.096889 30408 ts_tablet_manager.cc:1367] T 6a3c10a388d8469bb04a21b48b4aafaa P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:02.097266 30408 raft_consensus.cc:348] T 6a3c10a388d8469bb04a21b48b4aafaa P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:02.097364 30408 raft_consensus.cc:374] T 6a3c10a388d8469bb04a21b48b4aafaa P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:02.097393 30408 raft_consensus.cc:729] T 6a3c10a388d8469bb04a21b48b4aafaa P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:02.097540 30409 tablet_bootstrap.cc:492] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:02.097506 30408 consensus_queue.cc:260] T 6a3c10a388d8469bb04a21b48b4aafaa P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:02.097774 30408 ts_tablet_manager.cc:1398] T 6a3c10a388d8469bb04a21b48b4aafaa P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:02.097855 30406 tablet_bootstrap.cc:492] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:02.097927 30406 ts_tablet_manager.cc:1367] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 20:46:02.097923 29163 tablet_service.cc:1467] Processing CreateTablet for tablet fbd8fbbb18334242834ec7fa39184204 (DEFAULT_TABLE table=rebalance_test_table_4 [id=b684c3e68fbe4841bf5ca5674beb20f1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:02.098131 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbd8fbbb18334242834ec7fa39184204. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:02.098380 30409 tablet_bootstrap.cc:654] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:02.098328 30406 raft_consensus.cc:348] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:02.098415 30406 raft_consensus.cc:374] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:02.098443 30406 raft_consensus.cc:729] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:02.098675 30406 consensus_queue.cc:260] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:02.098927 30406 ts_tablet_manager.cc:1398] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:02.099340 30409 tablet_bootstrap.cc:492] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:02.099392 30409 ts_tablet_manager.cc:1367] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:02.099678 30409 raft_consensus.cc:348] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:02.099749 30409 raft_consensus.cc:374] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:02.099768 30409 raft_consensus.cc:729] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:02.099860 30409 consensus_queue.cc:260] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:02.099987 30407 tablet_bootstrap.cc:492] T fbd8fbbb18334242834ec7fa39184204 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:02.100128 30409 ts_tablet_manager.cc:1398] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:02.100529 29555 tablet_service.cc:1467] Processing CreateTablet for tablet fbd8fbbb18334242834ec7fa39184204 (DEFAULT_TABLE table=rebalance_test_table_4 [id=b684c3e68fbe4841bf5ca5674beb20f1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:02.100757 29555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbd8fbbb18334242834ec7fa39184204. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:02.100818 30407 tablet_bootstrap.cc:654] T fbd8fbbb18334242834ec7fa39184204 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:02.101940 30407 tablet_bootstrap.cc:492] T fbd8fbbb18334242834ec7fa39184204 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:02.101994 30407 ts_tablet_manager.cc:1367] T fbd8fbbb18334242834ec7fa39184204 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:02.102329 30407 raft_consensus.cc:348] T fbd8fbbb18334242834ec7fa39184204 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:02.102416 30407 raft_consensus.cc:374] T fbd8fbbb18334242834ec7fa39184204 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:02.102447 30407 raft_consensus.cc:729] T fbd8fbbb18334242834ec7fa39184204 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:02.102572 30407 consensus_queue.cc:260] T fbd8fbbb18334242834ec7fa39184204 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:02.102811 30407 ts_tablet_manager.cc:1398] T fbd8fbbb18334242834ec7fa39184204 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:02.103154 30408 tablet_bootstrap.cc:492] T fbd8fbbb18334242834ec7fa39184204 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:02.103957 30408 tablet_bootstrap.cc:654] T fbd8fbbb18334242834ec7fa39184204 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:02.104949 30408 tablet_bootstrap.cc:492] T fbd8fbbb18334242834ec7fa39184204 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:02.105017 30408 ts_tablet_manager.cc:1367] T fbd8fbbb18334242834ec7fa39184204 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:02.105353 30408 raft_consensus.cc:348] T fbd8fbbb18334242834ec7fa39184204 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:02.105427 30408 raft_consensus.cc:374] T fbd8fbbb18334242834ec7fa39184204 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:02.105454 30408 raft_consensus.cc:729] T fbd8fbbb18334242834ec7fa39184204 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:02.105561 30408 consensus_queue.cc:260] T fbd8fbbb18334242834ec7fa39184204 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:02.105779 30408 ts_tablet_manager.cc:1398] T fbd8fbbb18334242834ec7fa39184204 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:02.133066 30173 raft_consensus.cc:482] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:02.133165 30173 raft_consensus.cc:504] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:02.133508 30173 leader_election.cc:288] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:02.133852 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a3c10a388d8469bb04a21b48b4aafaa" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:02.133873 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a3c10a388d8469bb04a21b48b4aafaa" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:02.133988 29175 raft_consensus.cc:2454] T 6a3c10a388d8469bb04a21b48b4aafaa P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:02.133988 29575 raft_consensus.cc:2454] T 6a3c10a388d8469bb04a21b48b4aafaa P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:02.134271 29249 leader_election.cc:302] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:02.134443 30173 raft_consensus.cc:2781] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:02.134517 30173 raft_consensus.cc:482] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:02.134543 30173 raft_consensus.cc:3037] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:02.135254 30173 raft_consensus.cc:504] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:02.135545 30173 leader_election.cc:288] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:02.135846 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a3c10a388d8469bb04a21b48b4aafaa" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:02.135859 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a3c10a388d8469bb04a21b48b4aafaa" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:02.135955 29575 raft_consensus.cc:3037] T 6a3c10a388d8469bb04a21b48b4aafaa P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:02.135965 29175 raft_consensus.cc:3037] T 6a3c10a388d8469bb04a21b48b4aafaa P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:02.136763 29575 raft_consensus.cc:2454] T 6a3c10a388d8469bb04a21b48b4aafaa P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:02.136773 29175 raft_consensus.cc:2454] T 6a3c10a388d8469bb04a21b48b4aafaa P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:02.137022 29249 leader_election.cc:302] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:02.137162 30173 raft_consensus.cc:2781] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:02.137332 30173 raft_consensus.cc:686] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:02.137455 30173 consensus_queue.cc:237] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:02.138844 29039 catalog_manager.cc:5484] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:02.192466 30173 raft_consensus.cc:482] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:02.192569 30173 raft_consensus.cc:504] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:02.192932 30173 leader_election.cc:288] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:02.193281 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44010b796bae4b97ab7f17ee5cdffc30" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:02.193284 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44010b796bae4b97ab7f17ee5cdffc30" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:02.193400 29446 raft_consensus.cc:2454] T 44010b796bae4b97ab7f17ee5cdffc30 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:02.193408 29175 raft_consensus.cc:2454] T 44010b796bae4b97ab7f17ee5cdffc30 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:02.193670 29250 leader_election.cc:302] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:02.193833 30173 raft_consensus.cc:2781] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:02.193892 30173 raft_consensus.cc:482] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:02.193926 30173 raft_consensus.cc:3037] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:02.194669 30173 raft_consensus.cc:504] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:02.194954 30173 leader_election.cc:288] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:02.195252 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44010b796bae4b97ab7f17ee5cdffc30" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:02.195252 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44010b796bae4b97ab7f17ee5cdffc30" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:02.195365 29175 raft_consensus.cc:3037] T 44010b796bae4b97ab7f17ee5cdffc30 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:02.195365 29446 raft_consensus.cc:3037] T 44010b796bae4b97ab7f17ee5cdffc30 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:02.196223 29175 raft_consensus.cc:2454] T 44010b796bae4b97ab7f17ee5cdffc30 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:02.196223 29446 raft_consensus.cc:2454] T 44010b796bae4b97ab7f17ee5cdffc30 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:02.196491 29251 leader_election.cc:302] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:02.196661 30173 raft_consensus.cc:2781] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:02.196740 30173 raft_consensus.cc:686] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:02.196902 30173 consensus_queue.cc:237] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:02.198302 29039 catalog_manager.cc:5484] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:02.365567 30335 raft_consensus.cc:482] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:02.365684 30335 raft_consensus.cc:504] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:02.366122 30335 leader_election.cc:288] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:02.366549 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbd8fbbb18334242834ec7fa39184204" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:02.366554 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbd8fbbb18334242834ec7fa39184204" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:02.366678 29575 raft_consensus.cc:2454] T fbd8fbbb18334242834ec7fa39184204 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:02.366696 29175 raft_consensus.cc:2454] T fbd8fbbb18334242834ec7fa39184204 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:02.366984 29380 leader_election.cc:302] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:02.367144 30335 raft_consensus.cc:2781] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:02.367209 30335 raft_consensus.cc:482] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:02.367251 30335 raft_consensus.cc:3037] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:02.368064 30335 raft_consensus.cc:504] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:02.368363 30335 leader_election.cc:288] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:02.368667 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbd8fbbb18334242834ec7fa39184204" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:02.368674 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbd8fbbb18334242834ec7fa39184204" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:02.368762 29175 raft_consensus.cc:3037] T fbd8fbbb18334242834ec7fa39184204 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:02.368767 29575 raft_consensus.cc:3037] T fbd8fbbb18334242834ec7fa39184204 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:02.369442 29175 raft_consensus.cc:2454] T fbd8fbbb18334242834ec7fa39184204 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:02.369452 29575 raft_consensus.cc:2454] T fbd8fbbb18334242834ec7fa39184204 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:02.369686 29380 leader_election.cc:302] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:02.369860 30335 raft_consensus.cc:2781] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:02.369948 30335 raft_consensus.cc:686] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:02.370079 30335 consensus_queue.cc:237] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:02.371555 29039 catalog_manager.cc:5484] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
W20240502 20:46:02.485596 30473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:02.485800 30473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:02.488687 30473 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
I20240502 20:46:02.558933 29175 raft_consensus.cc:1261] T fbd8fbbb18334242834ec7fa39184204 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:02.559351 29575 raft_consensus.cc:1261] T 6a3c10a388d8469bb04a21b48b4aafaa P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:02.559389 29576 raft_consensus.cc:1261] T fbd8fbbb18334242834ec7fa39184204 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:02.559931 30173 consensus_queue.cc:1035] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:02.559935 30419 consensus_queue.cc:1035] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:02.560230 30419 consensus_queue.cc:1035] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:02.563253 29175 raft_consensus.cc:1261] T 6a3c10a388d8469bb04a21b48b4aafaa P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:02.563819 30173 consensus_queue.cc:1035] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:02.568178 29183 raft_consensus.cc:1261] T 44010b796bae4b97ab7f17ee5cdffc30 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:02.568557 30173 consensus_queue.cc:1035] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:02.573632 29446 raft_consensus.cc:1261] T 44010b796bae4b97ab7f17ee5cdffc30 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:02.574108 30469 consensus_queue.cc:1035] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 95.1669 ms
time per row: 0.0475834 ms
I20240502 20:46:02.623883 29039 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalance_test_table_5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:02.628918 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 760d15b2acee4322a26f1f2a965f22cc (DEFAULT_TABLE table=rebalance_test_table_5 [id=fd2f36435a614aa98d7f33e9646f33a3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:02.629223 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 760d15b2acee4322a26f1f2a965f22cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:02.629555 29163 tablet_service.cc:1467] Processing CreateTablet for tablet d4a2c7b197f948a0b236d5e05e2749b9 (DEFAULT_TABLE table=rebalance_test_table_5 [id=fd2f36435a614aa98d7f33e9646f33a3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:02.629810 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4a2c7b197f948a0b236d5e05e2749b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:02.631260 30528 tablet_bootstrap.cc:492] T 760d15b2acee4322a26f1f2a965f22cc P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:02.631989 29424 tablet_service.cc:1467] Processing CreateTablet for tablet 760d15b2acee4322a26f1f2a965f22cc (DEFAULT_TABLE table=rebalance_test_table_5 [id=fd2f36435a614aa98d7f33e9646f33a3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:02.632076 29294 tablet_service.cc:1467] Processing CreateTablet for tablet d4a2c7b197f948a0b236d5e05e2749b9 (DEFAULT_TABLE table=rebalance_test_table_5 [id=fd2f36435a614aa98d7f33e9646f33a3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:02.632066 29426 tablet_service.cc:1467] Processing CreateTablet for tablet 9d2cd90615dc4d1894ffe5e80914bdc0 (DEFAULT_TABLE table=rebalance_test_table_5 [id=fd2f36435a614aa98d7f33e9646f33a3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:02.632174 30528 tablet_bootstrap.cc:654] T 760d15b2acee4322a26f1f2a965f22cc P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:02.632237 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 760d15b2acee4322a26f1f2a965f22cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:02.632263 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4a2c7b197f948a0b236d5e05e2749b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:02.632493 29425 tablet_service.cc:1467] Processing CreateTablet for tablet d4a2c7b197f948a0b236d5e05e2749b9 (DEFAULT_TABLE table=rebalance_test_table_5 [id=fd2f36435a614aa98d7f33e9646f33a3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:02.632701 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4a2c7b197f948a0b236d5e05e2749b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:02.634184 29426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d2cd90615dc4d1894ffe5e80914bdc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:02.634344 30529 tablet_bootstrap.cc:492] T d4a2c7b197f948a0b236d5e05e2749b9 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:02.634558 30531 tablet_bootstrap.cc:492] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:02.635080 30529 tablet_bootstrap.cc:654] T d4a2c7b197f948a0b236d5e05e2749b9 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:02.635305 30531 tablet_bootstrap.cc:654] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:02.635993 30529 tablet_bootstrap.cc:492] T d4a2c7b197f948a0b236d5e05e2749b9 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:02.636076 30529 ts_tablet_manager.cc:1367] T d4a2c7b197f948a0b236d5e05e2749b9 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:02.636456 30531 tablet_bootstrap.cc:492] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:02.636535 30531 ts_tablet_manager.cc:1367] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:02.636484 30529 raft_consensus.cc:348] T d4a2c7b197f948a0b236d5e05e2749b9 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:02.636521 29555 tablet_service.cc:1467] Processing CreateTablet for tablet 760d15b2acee4322a26f1f2a965f22cc (DEFAULT_TABLE table=rebalance_test_table_5 [id=fd2f36435a614aa98d7f33e9646f33a3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:02.636584 30529 raft_consensus.cc:374] T d4a2c7b197f948a0b236d5e05e2749b9 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:02.636548 29557 tablet_service.cc:1467] Processing CreateTablet for tablet 9d2cd90615dc4d1894ffe5e80914bdc0 (DEFAULT_TABLE table=rebalance_test_table_5 [id=fd2f36435a614aa98d7f33e9646f33a3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:02.636613 30529 raft_consensus.cc:729] T d4a2c7b197f948a0b236d5e05e2749b9 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:02.636775 29555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 760d15b2acee4322a26f1f2a965f22cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:02.636744 30529 consensus_queue.cc:260] T d4a2c7b197f948a0b236d5e05e2749b9 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:02.636930 30531 raft_consensus.cc:348] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:02.637023 30531 raft_consensus.cc:374] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:02.637037 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d2cd90615dc4d1894ffe5e80914bdc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:02.637053 30531 raft_consensus.cc:729] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:02.637079 30529 ts_tablet_manager.cc:1398] T d4a2c7b197f948a0b236d5e05e2749b9 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:02.637156 30531 consensus_queue.cc:260] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:02.637419 30531 ts_tablet_manager.cc:1398] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:02.637560 30531 tablet_bootstrap.cc:492] T d4a2c7b197f948a0b236d5e05e2749b9 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:02.638312 30531 tablet_bootstrap.cc:654] T d4a2c7b197f948a0b236d5e05e2749b9 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:02.638418 30534 tablet_bootstrap.cc:492] T 760d15b2acee4322a26f1f2a965f22cc P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:02.639468 30534 tablet_bootstrap.cc:654] T 760d15b2acee4322a26f1f2a965f22cc P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:02.640393 30534 tablet_bootstrap.cc:492] T 760d15b2acee4322a26f1f2a965f22cc P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:02.640480 30534 ts_tablet_manager.cc:1367] T 760d15b2acee4322a26f1f2a965f22cc P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:02.640657 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 9d2cd90615dc4d1894ffe5e80914bdc0 (DEFAULT_TABLE table=rebalance_test_table_5 [id=fd2f36435a614aa98d7f33e9646f33a3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:02.640890 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d2cd90615dc4d1894ffe5e80914bdc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:02.640861 30534 raft_consensus.cc:348] T 760d15b2acee4322a26f1f2a965f22cc P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:02.640933 30534 raft_consensus.cc:374] T 760d15b2acee4322a26f1f2a965f22cc P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:02.640951 30534 raft_consensus.cc:729] T 760d15b2acee4322a26f1f2a965f22cc P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:02.641059 30534 consensus_queue.cc:260] T 760d15b2acee4322a26f1f2a965f22cc P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:02.641279 30534 ts_tablet_manager.cc:1398] T 760d15b2acee4322a26f1f2a965f22cc P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:02.641430 30534 tablet_bootstrap.cc:492] T 9d2cd90615dc4d1894ffe5e80914bdc0 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:02.642391 30534 tablet_bootstrap.cc:654] T 9d2cd90615dc4d1894ffe5e80914bdc0 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:02.642925 30187 raft_consensus.cc:482] T d4a2c7b197f948a0b236d5e05e2749b9 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:02.643007 30187 raft_consensus.cc:504] T d4a2c7b197f948a0b236d5e05e2749b9 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:02.643191 30528 tablet_bootstrap.cc:492] T 760d15b2acee4322a26f1f2a965f22cc P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:02.643288 30528 ts_tablet_manager.cc:1367] T 760d15b2acee4322a26f1f2a965f22cc P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 20:46:02.643352 30187 leader_election.cc:288] T d4a2c7b197f948a0b236d5e05e2749b9 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:02.643730 30528 raft_consensus.cc:348] T 760d15b2acee4322a26f1f2a965f22cc P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:02.643844 30528 raft_consensus.cc:374] T 760d15b2acee4322a26f1f2a965f22cc P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:02.643882 30528 raft_consensus.cc:729] T 760d15b2acee4322a26f1f2a965f22cc P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:02.643908 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4a2c7b197f948a0b236d5e05e2749b9" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:02.643990 30528 consensus_queue.cc:260] T 760d15b2acee4322a26f1f2a965f22cc P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:02.644193 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4a2c7b197f948a0b236d5e05e2749b9" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:02.644235 30528 ts_tablet_manager.cc:1398] T 760d15b2acee4322a26f1f2a965f22cc P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:02.644389 30528 tablet_bootstrap.cc:492] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
W20240502 20:46:02.644594 29119 leader_election.cc:341] T d4a2c7b197f948a0b236d5e05e2749b9 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): Illegal state: must be running to vote when last-logged opid is not known
I20240502 20:46:02.644728 30531 tablet_bootstrap.cc:492] T d4a2c7b197f948a0b236d5e05e2749b9 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
W20240502 20:46:02.644735 29119 leader_election.cc:341] T d4a2c7b197f948a0b236d5e05e2749b9 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): Illegal state: must be running to vote when last-logged opid is not known
I20240502 20:46:02.644780 29119 leader_election.cc:302] T d4a2c7b197f948a0b236d5e05e2749b9 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7796ea3fba294d17a34386873969431d; no voters: 09258ac095664fa99caf0707262cde13, dbcdc6ac635b44b185c3eba63f66e9d6
I20240502 20:46:02.644917 30531 ts_tablet_manager.cc:1367] T d4a2c7b197f948a0b236d5e05e2749b9 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 20:46:02.644919 30187 raft_consensus.cc:2726] T d4a2c7b197f948a0b236d5e05e2749b9 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 20:46:02.645200 30528 tablet_bootstrap.cc:654] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:02.645412 30534 tablet_bootstrap.cc:492] T 9d2cd90615dc4d1894ffe5e80914bdc0 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:02.645509 30534 ts_tablet_manager.cc:1367] T 9d2cd90615dc4d1894ffe5e80914bdc0 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:02.645448 30531 raft_consensus.cc:348] T d4a2c7b197f948a0b236d5e05e2749b9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:02.645676 30531 raft_consensus.cc:374] T d4a2c7b197f948a0b236d5e05e2749b9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:02.645748 30531 raft_consensus.cc:729] T d4a2c7b197f948a0b236d5e05e2749b9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:02.645910 30534 raft_consensus.cc:348] T 9d2cd90615dc4d1894ffe5e80914bdc0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:02.645927 30531 consensus_queue.cc:260] T d4a2c7b197f948a0b236d5e05e2749b9 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:02.646008 30534 raft_consensus.cc:374] T 9d2cd90615dc4d1894ffe5e80914bdc0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:02.646039 30534 raft_consensus.cc:729] T 9d2cd90615dc4d1894ffe5e80914bdc0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:02.646143 30534 consensus_queue.cc:260] T 9d2cd90615dc4d1894ffe5e80914bdc0 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:02.646283 30531 ts_tablet_manager.cc:1398] T d4a2c7b197f948a0b236d5e05e2749b9 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:02.646397 30534 ts_tablet_manager.cc:1398] T 9d2cd90615dc4d1894ffe5e80914bdc0 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:02.646553 30531 tablet_bootstrap.cc:492] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:02.647083 30528 tablet_bootstrap.cc:492] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:02.647169 30528 ts_tablet_manager.cc:1367] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:02.647400 30531 tablet_bootstrap.cc:654] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:02.647564 30528 raft_consensus.cc:348] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:02.647678 30528 raft_consensus.cc:374] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:02.647708 30528 raft_consensus.cc:729] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:02.647859 30528 consensus_queue.cc:260] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:02.648125 30528 ts_tablet_manager.cc:1398] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:02.648438 30528 tablet_bootstrap.cc:492] T 9d2cd90615dc4d1894ffe5e80914bdc0 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:02.649183 30531 tablet_bootstrap.cc:492] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:02.649241 30528 tablet_bootstrap.cc:654] T 9d2cd90615dc4d1894ffe5e80914bdc0 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:02.649361 30531 ts_tablet_manager.cc:1367] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:02.649861 30531 raft_consensus.cc:348] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:02.650004 30531 raft_consensus.cc:374] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:02.650051 30531 raft_consensus.cc:729] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:02.650173 30531 consensus_queue.cc:260] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:02.650337 30528 tablet_bootstrap.cc:492] T 9d2cd90615dc4d1894ffe5e80914bdc0 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:02.650413 30528 ts_tablet_manager.cc:1367] T 9d2cd90615dc4d1894ffe5e80914bdc0 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:02.650589 30531 ts_tablet_manager.cc:1398] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:02.650810 30528 raft_consensus.cc:348] T 9d2cd90615dc4d1894ffe5e80914bdc0 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:02.650888 30528 raft_consensus.cc:374] T 9d2cd90615dc4d1894ffe5e80914bdc0 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:02.650910 30528 raft_consensus.cc:729] T 9d2cd90615dc4d1894ffe5e80914bdc0 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:02.651038 30528 consensus_queue.cc:260] T 9d2cd90615dc4d1894ffe5e80914bdc0 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:02.651297 30528 ts_tablet_manager.cc:1398] T 9d2cd90615dc4d1894ffe5e80914bdc0 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:02.667743 30513 raft_consensus.cc:482] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:02.667846 30513 raft_consensus.cc:504] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:02.668228 30513 leader_election.cc:288] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:02.668571 29183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4a2c7b197f948a0b236d5e05e2749b9" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:02.668579 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4a2c7b197f948a0b236d5e05e2749b9" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:02.668696 29446 raft_consensus.cc:2454] T d4a2c7b197f948a0b236d5e05e2749b9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:02.668710 29183 raft_consensus.cc:2454] T d4a2c7b197f948a0b236d5e05e2749b9 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:02.668964 29250 leader_election.cc:302] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:02.669126 30513 raft_consensus.cc:2781] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:02.669193 30513 raft_consensus.cc:482] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:02.669236 30513 raft_consensus.cc:3037] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:02.670008 30513 raft_consensus.cc:504] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:02.670322 30513 leader_election.cc:288] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:02.670606 29183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4a2c7b197f948a0b236d5e05e2749b9" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:02.670658 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4a2c7b197f948a0b236d5e05e2749b9" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:02.670727 29183 raft_consensus.cc:3037] T d4a2c7b197f948a0b236d5e05e2749b9 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:02.670753 29446 raft_consensus.cc:3037] T d4a2c7b197f948a0b236d5e05e2749b9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:02.671604 29183 raft_consensus.cc:2454] T d4a2c7b197f948a0b236d5e05e2749b9 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:02.671604 29446 raft_consensus.cc:2454] T d4a2c7b197f948a0b236d5e05e2749b9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:02.671880 29250 leader_election.cc:302] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:02.672067 30513 raft_consensus.cc:2781] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:02.672138 30513 raft_consensus.cc:686] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:02.672266 30513 consensus_queue.cc:237] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:02.673684 29033 catalog_manager.cc:5484] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:02.806748 30419 raft_consensus.cc:482] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:02.806869 30419 raft_consensus.cc:504] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:02.807238 30419 leader_election.cc:288] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:02.807616 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "760d15b2acee4322a26f1f2a965f22cc" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:02.807646 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "760d15b2acee4322a26f1f2a965f22cc" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:02.807741 29575 raft_consensus.cc:2454] T 760d15b2acee4322a26f1f2a965f22cc P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:02.807783 29315 raft_consensus.cc:2454] T 760d15b2acee4322a26f1f2a965f22cc P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:02.807996 29380 leader_election.cc:302] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:02.808169 30419 raft_consensus.cc:2781] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:02.808228 30419 raft_consensus.cc:482] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:02.808254 30419 raft_consensus.cc:3037] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:02.811120 30419 raft_consensus.cc:504] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:02.811424 30419 leader_election.cc:288] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:02.811753 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "760d15b2acee4322a26f1f2a965f22cc" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:02.811756 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "760d15b2acee4322a26f1f2a965f22cc" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:02.811854 29575 raft_consensus.cc:3037] T 760d15b2acee4322a26f1f2a965f22cc P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:02.811859 29315 raft_consensus.cc:3037] T 760d15b2acee4322a26f1f2a965f22cc P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:02.812575 29315 raft_consensus.cc:2454] T 760d15b2acee4322a26f1f2a965f22cc P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:02.812635 29575 raft_consensus.cc:2454] T 760d15b2acee4322a26f1f2a965f22cc P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:02.812855 29381 leader_election.cc:302] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:02.813009 30419 raft_consensus.cc:2781] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:02.813098 30419 raft_consensus.cc:686] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:02.813225 30419 consensus_queue.cc:237] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:02.814742 29033 catalog_manager.cc:5484] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:02.964792 30419 raft_consensus.cc:482] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:02.964902 30419 raft_consensus.cc:504] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:02.965247 30419 leader_election.cc:288] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:02.965588 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d2cd90615dc4d1894ffe5e80914bdc0" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:02.965620 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d2cd90615dc4d1894ffe5e80914bdc0" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:02.965737 29315 raft_consensus.cc:2454] T 9d2cd90615dc4d1894ffe5e80914bdc0 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:02.965737 29575 raft_consensus.cc:2454] T 9d2cd90615dc4d1894ffe5e80914bdc0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:02.966014 29381 leader_election.cc:302] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:02.966192 30419 raft_consensus.cc:2781] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:02.966251 30419 raft_consensus.cc:482] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:02.966276 30419 raft_consensus.cc:3037] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:02.967111 30419 raft_consensus.cc:504] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:02.967412 30419 leader_election.cc:288] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:02.967705 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d2cd90615dc4d1894ffe5e80914bdc0" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:02.967823 29315 raft_consensus.cc:3037] T 9d2cd90615dc4d1894ffe5e80914bdc0 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:02.967909 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d2cd90615dc4d1894ffe5e80914bdc0" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:02.968011 29575 raft_consensus.cc:3037] T 9d2cd90615dc4d1894ffe5e80914bdc0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:02.968741 29315 raft_consensus.cc:2454] T 9d2cd90615dc4d1894ffe5e80914bdc0 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:02.968817 29575 raft_consensus.cc:2454] T 9d2cd90615dc4d1894ffe5e80914bdc0 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:02.968979 29381 leader_election.cc:302] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:02.969228 30419 raft_consensus.cc:2781] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:02.969339 30419 raft_consensus.cc:686] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:02.969503 30419 consensus_queue.cc:237] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:02.971108 29033 catalog_manager.cc:5484] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
W20240502 20:46:03.103711 30544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:03.103925 30544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:03.106835 30544 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_5'
I20240502 20:46:03.157127 30513 consensus_queue.cc:1035] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:03.161011 30513 consensus_queue.cc:1035] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:03.195197 29315 raft_consensus.cc:1261] T 9d2cd90615dc4d1894ffe5e80914bdc0 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:03.195737 30419 consensus_queue.cc:1035] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:03.198493 29575 raft_consensus.cc:1261] T 9d2cd90615dc4d1894ffe5e80914bdc0 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:03.198989 30419 consensus_queue.cc:1035] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:03.204602 29314 raft_consensus.cc:1261] T 760d15b2acee4322a26f1f2a965f22cc P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:03.204988 30335 consensus_queue.cc:1035] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:03.209679 29575 raft_consensus.cc:1261] T 760d15b2acee4322a26f1f2a965f22cc P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:03.210038 30335 consensus_queue.cc:1035] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:03.215185 30580 mvcc.cc:205] Tried to move back new op lower bound from 7023340598061010944 to 7023340596484206592. Current Snapshot: MvccSnapshot[applied={T|T < 7023340598061010944}]
I20240502 20:46:03.236723 30581 mvcc.cc:205] Tried to move back new op lower bound from 7023340598038847488 to 7023340597124247552. Current Snapshot: MvccSnapshot[applied={T|T < 7023340598038847488}]
INSERT report
rows total: 2000
time total: 102.247 ms
time per row: 0.0511237 ms
I20240502 20:46:03.252602 29035 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalance_test_table_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:03.257550 29163 tablet_service.cc:1467] Processing CreateTablet for tablet 56928692a7284f33bd34e4396925c570 (DEFAULT_TABLE table=rebalance_test_table_6 [id=85861b676eab48d695a81e1716f5dadc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:03.257627 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 56928692a7284f33bd34e4396925c570 (DEFAULT_TABLE table=rebalance_test_table_6 [id=85861b676eab48d695a81e1716f5dadc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:03.257858 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56928692a7284f33bd34e4396925c570. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:03.257894 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56928692a7284f33bd34e4396925c570. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:03.258232 29164 tablet_service.cc:1467] Processing CreateTablet for tablet 8ef44f7e7371449882d44860ad8fa9f0 (DEFAULT_TABLE table=rebalance_test_table_6 [id=85861b676eab48d695a81e1716f5dadc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:03.258443 29164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ef44f7e7371449882d44860ad8fa9f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:03.259802 30602 tablet_bootstrap.cc:492] T 56928692a7284f33bd34e4396925c570 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:03.259778 29162 tablet_service.cc:1467] Processing CreateTablet for tablet 4bfb24d76b434cd5b138f868b1f29508 (DEFAULT_TABLE table=rebalance_test_table_6 [id=85861b676eab48d695a81e1716f5dadc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:03.259972 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bfb24d76b434cd5b138f868b1f29508. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:03.260409 29557 tablet_service.cc:1467] Processing CreateTablet for tablet 56928692a7284f33bd34e4396925c570 (DEFAULT_TABLE table=rebalance_test_table_6 [id=85861b676eab48d695a81e1716f5dadc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:03.260568 30602 tablet_bootstrap.cc:654] T 56928692a7284f33bd34e4396925c570 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:03.260663 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56928692a7284f33bd34e4396925c570. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:03.260906 29426 tablet_service.cc:1467] Processing CreateTablet for tablet 8ef44f7e7371449882d44860ad8fa9f0 (DEFAULT_TABLE table=rebalance_test_table_6 [id=85861b676eab48d695a81e1716f5dadc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:03.261133 29426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ef44f7e7371449882d44860ad8fa9f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:03.261808 29556 tablet_service.cc:1467] Processing CreateTablet for tablet 4bfb24d76b434cd5b138f868b1f29508 (DEFAULT_TABLE table=rebalance_test_table_6 [id=85861b676eab48d695a81e1716f5dadc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:03.261986 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bfb24d76b434cd5b138f868b1f29508. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:03.262670 30604 tablet_bootstrap.cc:492] T 56928692a7284f33bd34e4396925c570 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:03.262816 29555 tablet_service.cc:1467] Processing CreateTablet for tablet 8ef44f7e7371449882d44860ad8fa9f0 (DEFAULT_TABLE table=rebalance_test_table_6 [id=85861b676eab48d695a81e1716f5dadc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:03.263031 29555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ef44f7e7371449882d44860ad8fa9f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:03.263053 29294 tablet_service.cc:1467] Processing CreateTablet for tablet 4bfb24d76b434cd5b138f868b1f29508 (DEFAULT_TABLE table=rebalance_test_table_6 [id=85861b676eab48d695a81e1716f5dadc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:03.263291 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bfb24d76b434cd5b138f868b1f29508. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:03.264708 30606 tablet_bootstrap.cc:492] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:03.265045 30605 tablet_bootstrap.cc:492] T 8ef44f7e7371449882d44860ad8fa9f0 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:03.265480 30606 tablet_bootstrap.cc:654] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:03.265733 30602 tablet_bootstrap.cc:492] T 56928692a7284f33bd34e4396925c570 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:03.265821 30605 tablet_bootstrap.cc:654] T 8ef44f7e7371449882d44860ad8fa9f0 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:03.265839 30602 ts_tablet_manager.cc:1367] T 56928692a7284f33bd34e4396925c570 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 20:46:03.266223 30602 raft_consensus.cc:348] T 56928692a7284f33bd34e4396925c570 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:03.266291 30606 tablet_bootstrap.cc:492] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:03.266337 30602 raft_consensus.cc:374] T 56928692a7284f33bd34e4396925c570 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:03.266381 30602 raft_consensus.cc:729] T 56928692a7284f33bd34e4396925c570 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:03.266446 30606 ts_tablet_manager.cc:1367] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:03.266544 30602 consensus_queue.cc:260] T 56928692a7284f33bd34e4396925c570 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:03.266830 30602 ts_tablet_manager.cc:1398] T 56928692a7284f33bd34e4396925c570 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:03.266942 30606 raft_consensus.cc:348] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:03.267069 30606 raft_consensus.cc:374] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:03.267103 30606 raft_consensus.cc:729] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:03.267133 30602 tablet_bootstrap.cc:492] T 8ef44f7e7371449882d44860ad8fa9f0 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:03.267241 30606 consensus_queue.cc:260] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:03.267495 30606 ts_tablet_manager.cc:1398] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:03.267633 30606 tablet_bootstrap.cc:492] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:03.267874 30602 tablet_bootstrap.cc:654] T 8ef44f7e7371449882d44860ad8fa9f0 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:03.268390 30606 tablet_bootstrap.cc:654] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:03.268580 30605 tablet_bootstrap.cc:492] T 8ef44f7e7371449882d44860ad8fa9f0 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:03.268666 30605 ts_tablet_manager.cc:1367] T 8ef44f7e7371449882d44860ad8fa9f0 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:03.269194 30602 tablet_bootstrap.cc:492] T 8ef44f7e7371449882d44860ad8fa9f0 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:03.269321 30602 ts_tablet_manager.cc:1367] T 8ef44f7e7371449882d44860ad8fa9f0 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 20:46:03.269526 30605 raft_consensus.cc:348] T 8ef44f7e7371449882d44860ad8fa9f0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:03.269632 30605 raft_consensus.cc:374] T 8ef44f7e7371449882d44860ad8fa9f0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:03.269670 30605 raft_consensus.cc:729] T 8ef44f7e7371449882d44860ad8fa9f0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:03.269780 30605 consensus_queue.cc:260] T 8ef44f7e7371449882d44860ad8fa9f0 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:03.269954 30602 raft_consensus.cc:348] T 8ef44f7e7371449882d44860ad8fa9f0 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:03.270052 30602 raft_consensus.cc:374] T 8ef44f7e7371449882d44860ad8fa9f0 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:03.270088 30602 raft_consensus.cc:729] T 8ef44f7e7371449882d44860ad8fa9f0 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:03.270148 30605 ts_tablet_manager.cc:1398] T 8ef44f7e7371449882d44860ad8fa9f0 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:03.270198 30602 consensus_queue.cc:260] T 8ef44f7e7371449882d44860ad8fa9f0 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:03.270511 30602 ts_tablet_manager.cc:1398] T 8ef44f7e7371449882d44860ad8fa9f0 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:03.270645 30602 tablet_bootstrap.cc:492] T 4bfb24d76b434cd5b138f868b1f29508 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:03.270794 30606 tablet_bootstrap.cc:492] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:03.270864 30606 ts_tablet_manager.cc:1367] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 20:46:03.271251 30606 raft_consensus.cc:348] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:03.271359 30606 raft_consensus.cc:374] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:03.271392 30606 raft_consensus.cc:729] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:03.271425 30602 tablet_bootstrap.cc:654] T 4bfb24d76b434cd5b138f868b1f29508 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:03.271517 30606 consensus_queue.cc:260] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:03.271827 30606 ts_tablet_manager.cc:1398] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:03.272619 30602 tablet_bootstrap.cc:492] T 4bfb24d76b434cd5b138f868b1f29508 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:03.272699 30602 ts_tablet_manager.cc:1367] T 4bfb24d76b434cd5b138f868b1f29508 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:03.273022 30602 raft_consensus.cc:348] T 4bfb24d76b434cd5b138f868b1f29508 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:03.273113 30602 raft_consensus.cc:374] T 4bfb24d76b434cd5b138f868b1f29508 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:03.273144 30602 raft_consensus.cc:729] T 4bfb24d76b434cd5b138f868b1f29508 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:03.273255 30602 consensus_queue.cc:260] T 4bfb24d76b434cd5b138f868b1f29508 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:03.273504 30602 ts_tablet_manager.cc:1398] T 4bfb24d76b434cd5b138f868b1f29508 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:03.286581 30604 tablet_bootstrap.cc:654] T 56928692a7284f33bd34e4396925c570 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:03.287521 30604 tablet_bootstrap.cc:492] T 56928692a7284f33bd34e4396925c570 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:03.287591 30604 ts_tablet_manager.cc:1367] T 56928692a7284f33bd34e4396925c570 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.025s user 0.001s sys 0.000s
I20240502 20:46:03.287940 30604 raft_consensus.cc:348] T 56928692a7284f33bd34e4396925c570 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:03.288043 30604 raft_consensus.cc:374] T 56928692a7284f33bd34e4396925c570 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:03.288082 30604 raft_consensus.cc:729] T 56928692a7284f33bd34e4396925c570 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:03.288194 30604 consensus_queue.cc:260] T 56928692a7284f33bd34e4396925c570 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:03.288434 30604 ts_tablet_manager.cc:1398] T 56928692a7284f33bd34e4396925c570 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:03.288616 30604 tablet_bootstrap.cc:492] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:03.289319 30604 tablet_bootstrap.cc:654] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:03.290721 30604 tablet_bootstrap.cc:492] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:03.290814 30604 ts_tablet_manager.cc:1367] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:03.291180 30604 raft_consensus.cc:348] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:03.291260 30604 raft_consensus.cc:374] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:03.291280 30604 raft_consensus.cc:729] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:03.291361 30604 consensus_queue.cc:260] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:03.291568 30604 ts_tablet_manager.cc:1398] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:03.291724 30604 tablet_bootstrap.cc:492] T 4bfb24d76b434cd5b138f868b1f29508 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:03.292562 30604 tablet_bootstrap.cc:654] T 4bfb24d76b434cd5b138f868b1f29508 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:03.292730 30591 raft_consensus.cc:482] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:03.292810 30591 raft_consensus.cc:504] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:03.293181 30591 leader_election.cc:288] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:03.293452 29183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bfb24d76b434cd5b138f868b1f29508" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:03.293507 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bfb24d76b434cd5b138f868b1f29508" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:03.293583 29183 raft_consensus.cc:2454] T 4bfb24d76b434cd5b138f868b1f29508 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
W20240502 20:46:03.293807 29249 leader_election.cc:341] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): Illegal state: must be running to vote when last-logged opid is not known
I20240502 20:46:03.293903 29251 leader_election.cc:302] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters: db17f26a73f3447b92aa9a947e3a7057
I20240502 20:46:03.294059 30591 raft_consensus.cc:2781] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:03.294111 30591 raft_consensus.cc:482] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:03.294152 30591 raft_consensus.cc:3037] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:03.294363 30604 tablet_bootstrap.cc:492] T 4bfb24d76b434cd5b138f868b1f29508 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:03.294436 30604 ts_tablet_manager.cc:1367] T 4bfb24d76b434cd5b138f868b1f29508 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:03.294795 30604 raft_consensus.cc:348] T 4bfb24d76b434cd5b138f868b1f29508 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:03.294816 30591 raft_consensus.cc:504] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:03.294888 30604 raft_consensus.cc:374] T 4bfb24d76b434cd5b138f868b1f29508 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:03.294915 30604 raft_consensus.cc:729] T 4bfb24d76b434cd5b138f868b1f29508 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:03.295023 30604 consensus_queue.cc:260] T 4bfb24d76b434cd5b138f868b1f29508 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:03.295090 30591 leader_election.cc:288] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:03.295341 30604 ts_tablet_manager.cc:1398] T 4bfb24d76b434cd5b138f868b1f29508 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:03.295500 29183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bfb24d76b434cd5b138f868b1f29508" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:03.295593 29183 raft_consensus.cc:3037] T 4bfb24d76b434cd5b138f868b1f29508 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:03.295678 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bfb24d76b434cd5b138f868b1f29508" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:03.295795 29575 raft_consensus.cc:3037] T 4bfb24d76b434cd5b138f868b1f29508 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:03.296391 29575 raft_consensus.cc:2454] T 4bfb24d76b434cd5b138f868b1f29508 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:03.296475 29183 raft_consensus.cc:2454] T 4bfb24d76b434cd5b138f868b1f29508 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:03.296634 29249 leader_election.cc:302] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:03.296846 30591 raft_consensus.cc:2781] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:03.296934 30591 raft_consensus.cc:686] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:03.297057 30591 consensus_queue.cc:237] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:03.298741 29035 catalog_manager.cc:5484] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:03.357898 30591 raft_consensus.cc:482] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:03.358008 30591 raft_consensus.cc:504] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:03.358379 30591 leader_election.cc:288] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:03.358709 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56928692a7284f33bd34e4396925c570" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:03.358758 29183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56928692a7284f33bd34e4396925c570" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:03.358832 29575 raft_consensus.cc:2454] T 56928692a7284f33bd34e4396925c570 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:03.358875 29183 raft_consensus.cc:2454] T 56928692a7284f33bd34e4396925c570 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:03.359097 29249 leader_election.cc:302] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:03.359282 30591 raft_consensus.cc:2781] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:03.359351 30591 raft_consensus.cc:482] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:03.359385 30591 raft_consensus.cc:3037] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:03.360167 30591 raft_consensus.cc:504] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:03.360464 30591 leader_election.cc:288] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:03.360749 29183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56928692a7284f33bd34e4396925c570" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:03.360785 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56928692a7284f33bd34e4396925c570" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:03.360868 29575 raft_consensus.cc:3037] T 56928692a7284f33bd34e4396925c570 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:03.360870 29183 raft_consensus.cc:3037] T 56928692a7284f33bd34e4396925c570 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:03.361747 29183 raft_consensus.cc:2454] T 56928692a7284f33bd34e4396925c570 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:03.361748 29575 raft_consensus.cc:2454] T 56928692a7284f33bd34e4396925c570 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:03.362006 29249 leader_election.cc:302] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:03.362346 30591 raft_consensus.cc:2781] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:03.362432 30591 raft_consensus.cc:686] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:03.362565 30591 consensus_queue.cc:237] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:03.364109 29035 catalog_manager.cc:5484] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:03.457346 30199 raft_consensus.cc:482] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:03.457460 30199 raft_consensus.cc:504] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:03.457881 30199 leader_election.cc:288] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:03.458247 29183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ef44f7e7371449882d44860ad8fa9f0" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:03.458400 29183 raft_consensus.cc:2454] T 8ef44f7e7371449882d44860ad8fa9f0 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:03.458689 29513 leader_election.cc:302] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:03.458860 30199 raft_consensus.cc:2781] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:03.458911 30199 raft_consensus.cc:482] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:03.458946 30199 raft_consensus.cc:3037] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:03.459728 30199 raft_consensus.cc:504] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:03.460072 30199 leader_election.cc:288] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Requested vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:03.460408 29183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ef44f7e7371449882d44860ad8fa9f0" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:03.460536 29183 raft_consensus.cc:3037] T 8ef44f7e7371449882d44860ad8fa9f0 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:03.461398 29183 raft_consensus.cc:2454] T 8ef44f7e7371449882d44860ad8fa9f0 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:03.461642 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ef44f7e7371449882d44860ad8fa9f0" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:03.461671 29513 leader_election.cc:302] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:03.461658 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ef44f7e7371449882d44860ad8fa9f0" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:03.461777 29444 raft_consensus.cc:2454] T 8ef44f7e7371449882d44860ad8fa9f0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:03.461843 30199 raft_consensus.cc:2781] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:03.462023 30199 raft_consensus.cc:686] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Becoming Leader. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Running, Role: LEADER
I20240502 20:46:03.462150 30199 consensus_queue.cc:237] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:03.463801 29035 catalog_manager.cc:5484] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: term changed from 0 to 1, leader changed from <none> to db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
W20240502 20:46:03.602473 30620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:03.602756 30620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:03.606885 30620 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_6'
I20240502 20:46:03.685595 29183 raft_consensus.cc:1261] T 56928692a7284f33bd34e4396925c570 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:03.685621 29575 raft_consensus.cc:1261] T 56928692a7284f33bd34e4396925c570 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:03.686057 30591 consensus_queue.cc:1035] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:03.686509 30173 consensus_queue.cc:1035] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:03.690727 29184 raft_consensus.cc:1261] T 8ef44f7e7371449882d44860ad8fa9f0 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:03.691089 30199 consensus_queue.cc:1035] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:03.691985 29444 raft_consensus.cc:3037] T 8ef44f7e7371449882d44860ad8fa9f0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:03.693169 29444 raft_consensus.cc:1261] T 8ef44f7e7371449882d44860ad8fa9f0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:03.694288 29175 raft_consensus.cc:1261] T 4bfb24d76b434cd5b138f868b1f29508 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:03.695408 30173 consensus_queue.cc:1035] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:03.696194 29576 raft_consensus.cc:1261] T 4bfb24d76b434cd5b138f868b1f29508 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:03.696415 30649 consensus_queue.cc:1035] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:03.697789 30513 consensus_queue.cc:1035] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:03.712723 30583 mvcc.cc:205] Tried to move back new op lower bound from 7023340600052125696 to 7023340598734233600. Current Snapshot: MvccSnapshot[applied={T|T < 7023340600052125696 or (T in {7023340600052125696})}]
INSERT report
rows total: 2000
time total: 93.9899 ms
time per row: 0.0469949 ms
I20240502 20:46:03.746853 29038 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalance_test_table_7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:03.751696 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 4a91eafc5ecf45b09f60d712884b7c79 (DEFAULT_TABLE table=rebalance_test_table_7 [id=cf326efc9cf1428897a47dd98c65f088]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:03.751991 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a91eafc5ecf45b09f60d712884b7c79. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:03.751977 29426 tablet_service.cc:1467] Processing CreateTablet for tablet 4a91eafc5ecf45b09f60d712884b7c79 (DEFAULT_TABLE table=rebalance_test_table_7 [id=cf326efc9cf1428897a47dd98c65f088]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:03.751977 29163 tablet_service.cc:1467] Processing CreateTablet for tablet 4a91eafc5ecf45b09f60d712884b7c79 (DEFAULT_TABLE table=rebalance_test_table_7 [id=cf326efc9cf1428897a47dd98c65f088]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:03.752220 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a91eafc5ecf45b09f60d712884b7c79. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:03.752219 29426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a91eafc5ecf45b09f60d712884b7c79. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:03.753880 30606 tablet_bootstrap.cc:492] T 4a91eafc5ecf45b09f60d712884b7c79 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:03.754166 30605 tablet_bootstrap.cc:492] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:03.754325 29162 tablet_service.cc:1467] Processing CreateTablet for tablet a8916d4762844444bae6fa6f930abbfd (DEFAULT_TABLE table=rebalance_test_table_7 [id=cf326efc9cf1428897a47dd98c65f088]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:03.754570 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8916d4762844444bae6fa6f930abbfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:03.754715 30606 tablet_bootstrap.cc:654] T 4a91eafc5ecf45b09f60d712884b7c79 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:03.754993 30605 tablet_bootstrap.cc:654] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:03.755229 29424 tablet_service.cc:1467] Processing CreateTablet for tablet dcbca797b2d14e76b03e379cfe132597 (DEFAULT_TABLE table=rebalance_test_table_7 [id=cf326efc9cf1428897a47dd98c65f088]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:03.755475 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcbca797b2d14e76b03e379cfe132597. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:03.755496 30602 tablet_bootstrap.cc:492] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:03.756095 30606 tablet_bootstrap.cc:492] T 4a91eafc5ecf45b09f60d712884b7c79 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:03.756172 30606 ts_tablet_manager.cc:1367] T 4a91eafc5ecf45b09f60d712884b7c79 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:03.756307 30602 tablet_bootstrap.cc:654] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:03.756549 29556 tablet_service.cc:1467] Processing CreateTablet for tablet a8916d4762844444bae6fa6f930abbfd (DEFAULT_TABLE table=rebalance_test_table_7 [id=cf326efc9cf1428897a47dd98c65f088]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:03.756590 30606 raft_consensus.cc:348] T 4a91eafc5ecf45b09f60d712884b7c79 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:03.756695 30606 raft_consensus.cc:374] T 4a91eafc5ecf45b09f60d712884b7c79 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:03.756726 30606 raft_consensus.cc:729] T 4a91eafc5ecf45b09f60d712884b7c79 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:03.756805 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8916d4762844444bae6fa6f930abbfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:03.756836 30606 consensus_queue.cc:260] T 4a91eafc5ecf45b09f60d712884b7c79 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:03.757054 29555 tablet_service.cc:1467] Processing CreateTablet for tablet dcbca797b2d14e76b03e379cfe132597 (DEFAULT_TABLE table=rebalance_test_table_7 [id=cf326efc9cf1428897a47dd98c65f088]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:03.757103 30606 ts_tablet_manager.cc:1398] T 4a91eafc5ecf45b09f60d712884b7c79 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:03.757275 29555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcbca797b2d14e76b03e379cfe132597. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:03.758239 29426 tablet_service.cc:1467] Processing CreateTablet for tablet a8916d4762844444bae6fa6f930abbfd (DEFAULT_TABLE table=rebalance_test_table_7 [id=cf326efc9cf1428897a47dd98c65f088]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:03.758491 29426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8916d4762844444bae6fa6f930abbfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:03.759025 30604 tablet_bootstrap.cc:492] T a8916d4762844444bae6fa6f930abbfd P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:03.759361 29295 tablet_service.cc:1467] Processing CreateTablet for tablet dcbca797b2d14e76b03e379cfe132597 (DEFAULT_TABLE table=rebalance_test_table_7 [id=cf326efc9cf1428897a47dd98c65f088]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:03.759557 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcbca797b2d14e76b03e379cfe132597. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:03.759769 30604 tablet_bootstrap.cc:654] T a8916d4762844444bae6fa6f930abbfd P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:03.760898 30605 tablet_bootstrap.cc:492] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:03.760988 30605 ts_tablet_manager.cc:1367] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 20:46:03.761075 30602 tablet_bootstrap.cc:492] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:03.761166 30602 ts_tablet_manager.cc:1367] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20240502 20:46:03.761279 30606 tablet_bootstrap.cc:492] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:03.761402 30605 raft_consensus.cc:348] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:03.761497 30605 raft_consensus.cc:374] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:03.761525 30605 raft_consensus.cc:729] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:03.761572 30602 raft_consensus.cc:348] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:03.761672 30602 raft_consensus.cc:374] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:03.761647 30605 consensus_queue.cc:260] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:03.761700 30602 raft_consensus.cc:729] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:03.761806 30602 consensus_queue.cc:260] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:03.761920 30605 ts_tablet_manager.cc:1398] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:03.761976 30606 tablet_bootstrap.cc:654] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:03.762068 30605 tablet_bootstrap.cc:492] T dcbca797b2d14e76b03e379cfe132597 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:03.762084 30602 ts_tablet_manager.cc:1398] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:03.762549 30604 tablet_bootstrap.cc:492] T a8916d4762844444bae6fa6f930abbfd P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:03.762617 30604 ts_tablet_manager.cc:1367] T a8916d4762844444bae6fa6f930abbfd P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:03.763028 30605 tablet_bootstrap.cc:654] T dcbca797b2d14e76b03e379cfe132597 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:03.763005 30604 raft_consensus.cc:348] T a8916d4762844444bae6fa6f930abbfd P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:03.763105 30604 raft_consensus.cc:374] T a8916d4762844444bae6fa6f930abbfd P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:03.763144 30604 raft_consensus.cc:729] T a8916d4762844444bae6fa6f930abbfd P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:03.763271 30604 consensus_queue.cc:260] T a8916d4762844444bae6fa6f930abbfd P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:03.763545 30604 ts_tablet_manager.cc:1398] T a8916d4762844444bae6fa6f930abbfd P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:03.763700 30604 tablet_bootstrap.cc:492] T dcbca797b2d14e76b03e379cfe132597 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:03.764031 30605 tablet_bootstrap.cc:492] T dcbca797b2d14e76b03e379cfe132597 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:03.764115 30605 ts_tablet_manager.cc:1367] T dcbca797b2d14e76b03e379cfe132597 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:03.764498 30604 tablet_bootstrap.cc:654] T dcbca797b2d14e76b03e379cfe132597 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:03.764484 30605 raft_consensus.cc:348] T dcbca797b2d14e76b03e379cfe132597 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:03.764567 30605 raft_consensus.cc:374] T dcbca797b2d14e76b03e379cfe132597 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:03.764590 30606 tablet_bootstrap.cc:492] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:03.764602 30605 raft_consensus.cc:729] T dcbca797b2d14e76b03e379cfe132597 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:03.764662 30606 ts_tablet_manager.cc:1367] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 20:46:03.764719 30605 consensus_queue.cc:260] T dcbca797b2d14e76b03e379cfe132597 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:03.765054 30605 ts_tablet_manager.cc:1398] T dcbca797b2d14e76b03e379cfe132597 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:03.765069 30606 raft_consensus.cc:348] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:03.765153 30606 raft_consensus.cc:374] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:03.765180 30606 raft_consensus.cc:729] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:03.765201 30605 tablet_bootstrap.cc:492] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:03.765283 30606 consensus_queue.cc:260] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:03.765528 30606 ts_tablet_manager.cc:1398] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:03.765657 30602 tablet_bootstrap.cc:492] T a8916d4762844444bae6fa6f930abbfd P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:03.766129 30605 tablet_bootstrap.cc:654] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:03.766955 30604 tablet_bootstrap.cc:492] T dcbca797b2d14e76b03e379cfe132597 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:03.767030 30604 ts_tablet_manager.cc:1367] T dcbca797b2d14e76b03e379cfe132597 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:03.767125 30602 tablet_bootstrap.cc:654] T a8916d4762844444bae6fa6f930abbfd P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:03.767419 30604 raft_consensus.cc:348] T dcbca797b2d14e76b03e379cfe132597 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:03.767508 30604 raft_consensus.cc:374] T dcbca797b2d14e76b03e379cfe132597 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:03.767539 30604 raft_consensus.cc:729] T dcbca797b2d14e76b03e379cfe132597 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:03.767665 30604 consensus_queue.cc:260] T dcbca797b2d14e76b03e379cfe132597 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:03.767937 30604 ts_tablet_manager.cc:1398] T dcbca797b2d14e76b03e379cfe132597 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:03.769021 30602 tablet_bootstrap.cc:492] T a8916d4762844444bae6fa6f930abbfd P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:03.769101 30602 ts_tablet_manager.cc:1367] T a8916d4762844444bae6fa6f930abbfd P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 20:46:03.769441 30605 tablet_bootstrap.cc:492] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:03.769518 30605 ts_tablet_manager.cc:1367] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:03.769613 30602 raft_consensus.cc:348] T a8916d4762844444bae6fa6f930abbfd P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:03.769711 30602 raft_consensus.cc:374] T a8916d4762844444bae6fa6f930abbfd P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:03.769742 30602 raft_consensus.cc:729] T a8916d4762844444bae6fa6f930abbfd P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:03.769872 30602 consensus_queue.cc:260] T a8916d4762844444bae6fa6f930abbfd P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:03.769904 30605 raft_consensus.cc:348] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:03.769979 30605 raft_consensus.cc:374] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:03.770012 30605 raft_consensus.cc:729] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:03.770109 30605 consensus_queue.cc:260] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:03.770171 30602 ts_tablet_manager.cc:1398] T a8916d4762844444bae6fa6f930abbfd P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:03.770352 30605 ts_tablet_manager.cc:1398] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:03.771016 30659 raft_consensus.cc:482] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:03.771092 30659 raft_consensus.cc:504] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:03.771512 30659 leader_election.cc:288] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:03.771960 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a91eafc5ecf45b09f60d712884b7c79" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:03.772076 29182 raft_consensus.cc:2454] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:03.772289 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a91eafc5ecf45b09f60d712884b7c79" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:03.772446 29314 raft_consensus.cc:2454] T 4a91eafc5ecf45b09f60d712884b7c79 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:03.772703 29381 leader_election.cc:302] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:03.772920 30659 raft_consensus.cc:2781] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:03.772984 30659 raft_consensus.cc:482] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:03.773030 30659 raft_consensus.cc:3037] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:03.773674 30659 raft_consensus.cc:504] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:03.773950 30659 leader_election.cc:288] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:03.774333 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a91eafc5ecf45b09f60d712884b7c79" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:03.774492 29182 raft_consensus.cc:3037] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:03.774511 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a91eafc5ecf45b09f60d712884b7c79" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:03.774616 29314 raft_consensus.cc:3037] T 4a91eafc5ecf45b09f60d712884b7c79 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:03.775312 29182 raft_consensus.cc:2454] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:03.775355 29314 raft_consensus.cc:2454] T 4a91eafc5ecf45b09f60d712884b7c79 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:03.775619 29381 leader_election.cc:302] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:03.775795 30659 raft_consensus.cc:2781] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:03.775866 30659 raft_consensus.cc:686] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:03.775961 30659 consensus_queue.cc:237] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:03.777246 29034 catalog_manager.cc:5484] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:03.847893 30513 raft_consensus.cc:482] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:03.848021 30513 raft_consensus.cc:504] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:03.848431 30513 leader_election.cc:288] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:03.848734 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcbca797b2d14e76b03e379cfe132597" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:03.848867 29444 raft_consensus.cc:2454] T dcbca797b2d14e76b03e379cfe132597 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:03.849061 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcbca797b2d14e76b03e379cfe132597" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:03.849112 29250 leader_election.cc:302] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:03.849176 29575 raft_consensus.cc:2454] T dcbca797b2d14e76b03e379cfe132597 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:03.849285 30513 raft_consensus.cc:2781] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:03.849344 30513 raft_consensus.cc:482] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:03.849381 30513 raft_consensus.cc:3037] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:03.850198 30513 raft_consensus.cc:504] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:03.850525 30513 leader_election.cc:288] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:03.850832 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcbca797b2d14e76b03e379cfe132597" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:03.850840 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcbca797b2d14e76b03e379cfe132597" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:03.850935 29444 raft_consensus.cc:3037] T dcbca797b2d14e76b03e379cfe132597 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:03.850939 29575 raft_consensus.cc:3037] T dcbca797b2d14e76b03e379cfe132597 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:03.851598 29444 raft_consensus.cc:2454] T dcbca797b2d14e76b03e379cfe132597 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:03.851801 29575 raft_consensus.cc:2454] T dcbca797b2d14e76b03e379cfe132597 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:03.851845 29250 leader_election.cc:302] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:03.851999 30513 raft_consensus.cc:2781] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:03.852075 30513 raft_consensus.cc:686] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:03.852185 30513 consensus_queue.cc:237] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:03.853776 29034 catalog_manager.cc:5484] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:03.917467 30659 raft_consensus.cc:482] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:03.917582 30659 raft_consensus.cc:504] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:03.917973 30659 leader_election.cc:288] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:03.918282 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8916d4762844444bae6fa6f930abbfd" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:03.918282 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8916d4762844444bae6fa6f930abbfd" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:03.918418 29575 raft_consensus.cc:2454] T a8916d4762844444bae6fa6f930abbfd P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:03.918418 29182 raft_consensus.cc:2454] T a8916d4762844444bae6fa6f930abbfd P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:03.918717 29382 leader_election.cc:302] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:03.918864 30335 raft_consensus.cc:2781] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:03.918913 30335 raft_consensus.cc:482] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:03.918951 30335 raft_consensus.cc:3037] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:03.919651 30335 raft_consensus.cc:504] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:03.919924 30335 leader_election.cc:288] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:03.920192 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8916d4762844444bae6fa6f930abbfd" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:03.920297 29576 raft_consensus.cc:3037] T a8916d4762844444bae6fa6f930abbfd P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:03.920303 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8916d4762844444bae6fa6f930abbfd" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:03.920428 29182 raft_consensus.cc:3037] T a8916d4762844444bae6fa6f930abbfd P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:03.921048 29576 raft_consensus.cc:2454] T a8916d4762844444bae6fa6f930abbfd P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:03.921188 29182 raft_consensus.cc:2454] T a8916d4762844444bae6fa6f930abbfd P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:03.921294 29380 leader_election.cc:302] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:03.921453 30335 raft_consensus.cc:2781] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:03.921533 30335 raft_consensus.cc:686] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:03.921660 30335 consensus_queue.cc:237] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:03.923286 29034 catalog_manager.cc:5484] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
W20240502 20:46:04.044450 30672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:04.044644 30672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:04.047546 30672 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_7'
I20240502 20:46:04.121748 29182 raft_consensus.cc:1261] T a8916d4762844444bae6fa6f930abbfd P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:04.121945 29576 raft_consensus.cc:1261] T a8916d4762844444bae6fa6f930abbfd P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:04.122771 30659 consensus_queue.cc:1035] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:04.122920 30335 consensus_queue.cc:1035] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:04.124683 29444 raft_consensus.cc:1261] T dcbca797b2d14e76b03e379cfe132597 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:04.125231 29576 raft_consensus.cc:1261] T dcbca797b2d14e76b03e379cfe132597 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:04.125517 30513 consensus_queue.cc:1035] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:04.125800 30513 consensus_queue.cc:1035] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:04.126339 29314 raft_consensus.cc:1261] T 4a91eafc5ecf45b09f60d712884b7c79 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:04.126840 29184 raft_consensus.cc:1261] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:04.127426 30335 consensus_queue.cc:1035] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:04.127665 30335 consensus_queue.cc:1035] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:04.136760 30655 mvcc.cc:205] Tried to move back new op lower bound from 7023340601834688512 to 7023340601024507904. Current Snapshot: MvccSnapshot[applied={T|T < 7023340601834688512}]
INSERT report
rows total: 2000
time total: 94.5419 ms
time per row: 0.047271 ms
I20240502 20:46:04.183355 29038 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalance_test_table_8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:04.189090 29295 tablet_service.cc:1467] Processing CreateTablet for tablet f62df6fab4f248e7aae0a57a9ed58575 (DEFAULT_TABLE table=rebalance_test_table_8 [id=59a4763b0fad4ad084f27fcea88d8624]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:04.189368 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f62df6fab4f248e7aae0a57a9ed58575. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:04.189561 29555 tablet_service.cc:1467] Processing CreateTablet for tablet a76bead2ebc94234818f070ae4608d0f (DEFAULT_TABLE table=rebalance_test_table_8 [id=59a4763b0fad4ad084f27fcea88d8624]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:04.189841 29555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a76bead2ebc94234818f070ae4608d0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:04.190685 29556 tablet_service.cc:1467] Processing CreateTablet for tablet 89bea12cd67e4406a59d691037f385b6 (DEFAULT_TABLE table=rebalance_test_table_8 [id=59a4763b0fad4ad084f27fcea88d8624]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:04.190914 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89bea12cd67e4406a59d691037f385b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:04.191877 30606 tablet_bootstrap.cc:492] T f62df6fab4f248e7aae0a57a9ed58575 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:04.191860 29162 tablet_service.cc:1467] Processing CreateTablet for tablet f62df6fab4f248e7aae0a57a9ed58575 (DEFAULT_TABLE table=rebalance_test_table_8 [id=59a4763b0fad4ad084f27fcea88d8624]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:04.192111 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f62df6fab4f248e7aae0a57a9ed58575. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:04.192740 30606 tablet_bootstrap.cc:654] T f62df6fab4f248e7aae0a57a9ed58575 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:04.192888 29163 tablet_service.cc:1467] Processing CreateTablet for tablet a76bead2ebc94234818f070ae4608d0f (DEFAULT_TABLE table=rebalance_test_table_8 [id=59a4763b0fad4ad084f27fcea88d8624]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:04.193105 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a76bead2ebc94234818f070ae4608d0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:04.193457 29164 tablet_service.cc:1467] Processing CreateTablet for tablet 89bea12cd67e4406a59d691037f385b6 (DEFAULT_TABLE table=rebalance_test_table_8 [id=59a4763b0fad4ad084f27fcea88d8624]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:04.193682 29164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89bea12cd67e4406a59d691037f385b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:04.194106 30602 tablet_bootstrap.cc:492] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:04.195000 30602 tablet_bootstrap.cc:654] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:04.195494 30604 tablet_bootstrap.cc:492] T a76bead2ebc94234818f070ae4608d0f P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:04.195557 29295 tablet_service.cc:1467] Processing CreateTablet for tablet a76bead2ebc94234818f070ae4608d0f (DEFAULT_TABLE table=rebalance_test_table_8 [id=59a4763b0fad4ad084f27fcea88d8624]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:04.195794 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a76bead2ebc94234818f070ae4608d0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:04.196261 30604 tablet_bootstrap.cc:654] T a76bead2ebc94234818f070ae4608d0f P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:04.196655 30606 tablet_bootstrap.cc:492] T f62df6fab4f248e7aae0a57a9ed58575 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:04.196740 30606 ts_tablet_manager.cc:1367] T f62df6fab4f248e7aae0a57a9ed58575 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240502 20:46:04.197158 30606 raft_consensus.cc:348] T f62df6fab4f248e7aae0a57a9ed58575 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:04.197273 30606 raft_consensus.cc:374] T f62df6fab4f248e7aae0a57a9ed58575 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:04.197320 30606 raft_consensus.cc:729] T f62df6fab4f248e7aae0a57a9ed58575 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:04.197427 30606 consensus_queue.cc:260] T f62df6fab4f248e7aae0a57a9ed58575 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:04.197435 29426 tablet_service.cc:1467] Processing CreateTablet for tablet f62df6fab4f248e7aae0a57a9ed58575 (DEFAULT_TABLE table=rebalance_test_table_8 [id=59a4763b0fad4ad084f27fcea88d8624]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:04.197670 29426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f62df6fab4f248e7aae0a57a9ed58575. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:04.197705 30606 ts_tablet_manager.cc:1398] T f62df6fab4f248e7aae0a57a9ed58575 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:04.197865 30606 tablet_bootstrap.cc:492] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:04.198390 29424 tablet_service.cc:1467] Processing CreateTablet for tablet 89bea12cd67e4406a59d691037f385b6 (DEFAULT_TABLE table=rebalance_test_table_8 [id=59a4763b0fad4ad084f27fcea88d8624]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:04.198678 30606 tablet_bootstrap.cc:654] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:04.198671 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89bea12cd67e4406a59d691037f385b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:04.199565 30606 tablet_bootstrap.cc:492] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:04.199654 30606 ts_tablet_manager.cc:1367] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:04.200052 30606 raft_consensus.cc:348] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:04.200148 30606 raft_consensus.cc:374] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:04.200188 30606 raft_consensus.cc:729] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:04.200304 30606 consensus_queue.cc:260] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:04.200557 30606 ts_tablet_manager.cc:1398] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:04.200610 30605 tablet_bootstrap.cc:492] T 89bea12cd67e4406a59d691037f385b6 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:04.201133 30604 tablet_bootstrap.cc:492] T a76bead2ebc94234818f070ae4608d0f P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:04.201229 30604 ts_tablet_manager.cc:1367] T a76bead2ebc94234818f070ae4608d0f P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 20:46:04.201328 30602 tablet_bootstrap.cc:492] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:04.201336 30605 tablet_bootstrap.cc:654] T 89bea12cd67e4406a59d691037f385b6 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:04.201418 30602 ts_tablet_manager.cc:1367] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20240502 20:46:04.201659 30604 raft_consensus.cc:348] T a76bead2ebc94234818f070ae4608d0f P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:04.201764 30604 raft_consensus.cc:374] T a76bead2ebc94234818f070ae4608d0f P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:04.201802 30604 raft_consensus.cc:729] T a76bead2ebc94234818f070ae4608d0f P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:04.201810 30602 raft_consensus.cc:348] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:04.201908 30602 raft_consensus.cc:374] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:04.201949 30602 raft_consensus.cc:729] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:04.201920 30604 consensus_queue.cc:260] T a76bead2ebc94234818f070ae4608d0f P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:04.202061 30602 consensus_queue.cc:260] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:04.202172 30604 ts_tablet_manager.cc:1398] T a76bead2ebc94234818f070ae4608d0f P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:04.202335 30604 tablet_bootstrap.cc:492] T 89bea12cd67e4406a59d691037f385b6 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:04.202332 30602 ts_tablet_manager.cc:1398] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:04.202507 30602 tablet_bootstrap.cc:492] T a76bead2ebc94234818f070ae4608d0f P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:04.203354 30602 tablet_bootstrap.cc:654] T a76bead2ebc94234818f070ae4608d0f P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:04.204324 30602 tablet_bootstrap.cc:492] T a76bead2ebc94234818f070ae4608d0f P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:04.204406 30602 ts_tablet_manager.cc:1367] T a76bead2ebc94234818f070ae4608d0f P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:04.204519 30604 tablet_bootstrap.cc:654] T 89bea12cd67e4406a59d691037f385b6 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:04.204838 30602 raft_consensus.cc:348] T a76bead2ebc94234818f070ae4608d0f P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:04.204936 30602 raft_consensus.cc:374] T a76bead2ebc94234818f070ae4608d0f P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:04.204972 30602 raft_consensus.cc:729] T a76bead2ebc94234818f070ae4608d0f P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:04.205085 30602 consensus_queue.cc:260] T a76bead2ebc94234818f070ae4608d0f P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:04.205312 30602 ts_tablet_manager.cc:1398] T a76bead2ebc94234818f070ae4608d0f P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:04.205447 30602 tablet_bootstrap.cc:492] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:04.206075 30602 tablet_bootstrap.cc:654] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:04.207139 30602 tablet_bootstrap.cc:492] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:04.207211 30602 ts_tablet_manager.cc:1367] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:04.207580 30602 raft_consensus.cc:348] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:04.207682 30602 raft_consensus.cc:374] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:04.207713 30602 raft_consensus.cc:729] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:04.207825 30602 consensus_queue.cc:260] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:04.208052 30602 ts_tablet_manager.cc:1398] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:04.208498 30605 tablet_bootstrap.cc:492] T 89bea12cd67e4406a59d691037f385b6 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:04.208591 30605 ts_tablet_manager.cc:1367] T 89bea12cd67e4406a59d691037f385b6 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240502 20:46:04.208998 30605 raft_consensus.cc:348] T 89bea12cd67e4406a59d691037f385b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:04.209092 30605 raft_consensus.cc:374] T 89bea12cd67e4406a59d691037f385b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:04.209122 30605 raft_consensus.cc:729] T 89bea12cd67e4406a59d691037f385b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:04.209228 30605 consensus_queue.cc:260] T 89bea12cd67e4406a59d691037f385b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:04.209401 30604 tablet_bootstrap.cc:492] T 89bea12cd67e4406a59d691037f385b6 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:04.209463 30605 ts_tablet_manager.cc:1398] T 89bea12cd67e4406a59d691037f385b6 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:04.209553 30604 ts_tablet_manager.cc:1367] T 89bea12cd67e4406a59d691037f385b6 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 20:46:04.209609 30605 tablet_bootstrap.cc:492] T f62df6fab4f248e7aae0a57a9ed58575 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:04.210036 30604 raft_consensus.cc:348] T 89bea12cd67e4406a59d691037f385b6 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:04.210178 30604 raft_consensus.cc:374] T 89bea12cd67e4406a59d691037f385b6 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:04.210249 30604 raft_consensus.cc:729] T 89bea12cd67e4406a59d691037f385b6 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:04.210337 30605 tablet_bootstrap.cc:654] T f62df6fab4f248e7aae0a57a9ed58575 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:04.210448 30604 consensus_queue.cc:260] T 89bea12cd67e4406a59d691037f385b6 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:04.210806 30604 ts_tablet_manager.cc:1398] T 89bea12cd67e4406a59d691037f385b6 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:04.212661 30605 tablet_bootstrap.cc:492] T f62df6fab4f248e7aae0a57a9ed58575 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:04.212795 30605 ts_tablet_manager.cc:1367] T f62df6fab4f248e7aae0a57a9ed58575 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 20:46:04.213227 30605 raft_consensus.cc:348] T f62df6fab4f248e7aae0a57a9ed58575 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:04.213376 30605 raft_consensus.cc:374] T f62df6fab4f248e7aae0a57a9ed58575 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:04.213418 30605 raft_consensus.cc:729] T f62df6fab4f248e7aae0a57a9ed58575 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:04.213537 30605 consensus_queue.cc:260] T f62df6fab4f248e7aae0a57a9ed58575 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:04.213785 30605 ts_tablet_manager.cc:1398] T f62df6fab4f248e7aae0a57a9ed58575 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:04.238956 30591 raft_consensus.cc:482] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:04.239051 30591 raft_consensus.cc:504] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:04.239389 30591 leader_election.cc:288] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:04.239683 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a76bead2ebc94234818f070ae4608d0f" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:04.239706 29184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a76bead2ebc94234818f070ae4608d0f" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:04.239823 29575 raft_consensus.cc:2454] T a76bead2ebc94234818f070ae4608d0f P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:04.239822 29184 raft_consensus.cc:2454] T a76bead2ebc94234818f070ae4608d0f P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:04.240064 29249 leader_election.cc:302] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:04.240248 30591 raft_consensus.cc:2781] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:04.240306 30591 raft_consensus.cc:482] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:04.240331 30591 raft_consensus.cc:3037] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:04.241077 30591 raft_consensus.cc:504] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:04.241369 30591 leader_election.cc:288] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:04.241626 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a76bead2ebc94234818f070ae4608d0f" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:04.241667 29184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a76bead2ebc94234818f070ae4608d0f" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:04.241720 29575 raft_consensus.cc:3037] T a76bead2ebc94234818f070ae4608d0f P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:04.241767 29184 raft_consensus.cc:3037] T a76bead2ebc94234818f070ae4608d0f P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:04.242436 29575 raft_consensus.cc:2454] T a76bead2ebc94234818f070ae4608d0f P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:04.242627 29184 raft_consensus.cc:2454] T a76bead2ebc94234818f070ae4608d0f P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:04.242745 29249 leader_election.cc:302] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:04.242921 30591 raft_consensus.cc:2781] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:04.243002 30591 raft_consensus.cc:686] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:04.243127 30591 consensus_queue.cc:237] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:04.244804 29038 catalog_manager.cc:5484] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:04.273699 30187 raft_consensus.cc:482] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:04.273804 30187 raft_consensus.cc:504] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:04.274147 30187 leader_election.cc:288] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:04.274489 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f62df6fab4f248e7aae0a57a9ed58575" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:04.274561 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f62df6fab4f248e7aae0a57a9ed58575" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:04.274631 29314 raft_consensus.cc:2454] T f62df6fab4f248e7aae0a57a9ed58575 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:04.274682 29444 raft_consensus.cc:2454] T f62df6fab4f248e7aae0a57a9ed58575 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:04.274910 29119 leader_election.cc:302] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:04.275055 30187 raft_consensus.cc:2781] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:04.275130 30187 raft_consensus.cc:482] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:04.275185 30187 raft_consensus.cc:3037] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:04.275983 30187 raft_consensus.cc:504] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:04.276266 30187 leader_election.cc:288] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:04.276540 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f62df6fab4f248e7aae0a57a9ed58575" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:04.276634 29314 raft_consensus.cc:3037] T f62df6fab4f248e7aae0a57a9ed58575 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:04.276621 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f62df6fab4f248e7aae0a57a9ed58575" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:04.276727 29444 raft_consensus.cc:3037] T f62df6fab4f248e7aae0a57a9ed58575 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:04.277328 29314 raft_consensus.cc:2454] T f62df6fab4f248e7aae0a57a9ed58575 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:04.277401 29444 raft_consensus.cc:2454] T f62df6fab4f248e7aae0a57a9ed58575 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:04.277580 29119 leader_election.cc:302] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:04.277751 30187 raft_consensus.cc:2781] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:04.277842 30187 raft_consensus.cc:686] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:04.277967 30187 consensus_queue.cc:237] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:04.279482 29038 catalog_manager.cc:5484] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:04.377418 30187 raft_consensus.cc:482] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:04.377524 30187 raft_consensus.cc:504] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:04.377912 30187 leader_election.cc:288] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:04.378263 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89bea12cd67e4406a59d691037f385b6" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:04.378263 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89bea12cd67e4406a59d691037f385b6" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:04.378402 29444 raft_consensus.cc:2454] T 89bea12cd67e4406a59d691037f385b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:04.378402 29575 raft_consensus.cc:2454] T 89bea12cd67e4406a59d691037f385b6 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:04.378706 29119 leader_election.cc:302] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:04.378886 30187 raft_consensus.cc:2781] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:04.378957 30187 raft_consensus.cc:482] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:04.378998 30187 raft_consensus.cc:3037] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:04.379748 30187 raft_consensus.cc:504] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:04.380064 30187 leader_election.cc:288] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:04.380360 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89bea12cd67e4406a59d691037f385b6" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:04.380398 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89bea12cd67e4406a59d691037f385b6" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:04.380460 29575 raft_consensus.cc:3037] T 89bea12cd67e4406a59d691037f385b6 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:04.380502 29444 raft_consensus.cc:3037] T 89bea12cd67e4406a59d691037f385b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:04.381127 29575 raft_consensus.cc:2454] T 89bea12cd67e4406a59d691037f385b6 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:04.381237 29444 raft_consensus.cc:2454] T 89bea12cd67e4406a59d691037f385b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:04.381390 29118 leader_election.cc:302] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:04.381542 30187 raft_consensus.cc:2781] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:04.381620 30187 raft_consensus.cc:686] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:04.381747 30187 consensus_queue.cc:237] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:04.383332 29038 catalog_manager.cc:5484] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
W20240502 20:46:04.526578 30722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:04.526844 30722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:04.529814 30722 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_8'
I20240502 20:46:04.600013 29575 raft_consensus.cc:1261] T a76bead2ebc94234818f070ae4608d0f P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:04.600445 30513 consensus_queue.cc:1035] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:04.601945 29444 raft_consensus.cc:1261] T f62df6fab4f248e7aae0a57a9ed58575 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:04.602140 29184 raft_consensus.cc:1261] T a76bead2ebc94234818f070ae4608d0f P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:04.602645 29314 raft_consensus.cc:1261] T f62df6fab4f248e7aae0a57a9ed58575 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:04.604103 29576 raft_consensus.cc:1261] T 89bea12cd67e4406a59d691037f385b6 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:04.604467 30187 consensus_queue.cc:1035] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:04.605515 30654 consensus_queue.cc:1035] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:04.605836 30654 consensus_queue.cc:1035] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:04.606173 30513 consensus_queue.cc:1035] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:04.610821 29444 raft_consensus.cc:1261] T 89bea12cd67e4406a59d691037f385b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:04.612020 30187 consensus_queue.cc:1035] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 105.53 ms
time per row: 0.0527652 ms
I20240502 20:46:04.674285 29040 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalance_test_table_9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:04.679252 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 9acc12c9f0464e07965c05285fd334f0 (DEFAULT_TABLE table=rebalance_test_table_9 [id=4b510e9cf32a4794a9193f7ebac46f93]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:04.679294 29426 tablet_service.cc:1467] Processing CreateTablet for tablet 9acc12c9f0464e07965c05285fd334f0 (DEFAULT_TABLE table=rebalance_test_table_9 [id=4b510e9cf32a4794a9193f7ebac46f93]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:04.679546 29426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9acc12c9f0464e07965c05285fd334f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:04.679546 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9acc12c9f0464e07965c05285fd334f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:04.680390 29425 tablet_service.cc:1467] Processing CreateTablet for tablet 3943d49acd0341c1a923d900281f1591 (DEFAULT_TABLE table=rebalance_test_table_9 [id=4b510e9cf32a4794a9193f7ebac46f93]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:04.680567 29424 tablet_service.cc:1467] Processing CreateTablet for tablet 741da32f7c924490a5fa5933708adf30 (DEFAULT_TABLE table=rebalance_test_table_9 [id=4b510e9cf32a4794a9193f7ebac46f93]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:04.680641 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3943d49acd0341c1a923d900281f1591. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:04.680850 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 741da32f7c924490a5fa5933708adf30. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:04.681702 30606 tablet_bootstrap.cc:492] T 9acc12c9f0464e07965c05285fd334f0 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:04.681649 29555 tablet_service.cc:1467] Processing CreateTablet for tablet 741da32f7c924490a5fa5933708adf30 (DEFAULT_TABLE table=rebalance_test_table_9 [id=4b510e9cf32a4794a9193f7ebac46f93]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:04.681895 29555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 741da32f7c924490a5fa5933708adf30. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:04.682384 30606 tablet_bootstrap.cc:654] T 9acc12c9f0464e07965c05285fd334f0 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:04.682881 29556 tablet_service.cc:1467] Processing CreateTablet for tablet 9acc12c9f0464e07965c05285fd334f0 (DEFAULT_TABLE table=rebalance_test_table_9 [id=4b510e9cf32a4794a9193f7ebac46f93]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:04.683099 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9acc12c9f0464e07965c05285fd334f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:04.684700 30605 tablet_bootstrap.cc:492] T 9acc12c9f0464e07965c05285fd334f0 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:04.685313 30606 tablet_bootstrap.cc:492] T 9acc12c9f0464e07965c05285fd334f0 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:04.685382 30606 ts_tablet_manager.cc:1367] T 9acc12c9f0464e07965c05285fd334f0 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:04.685415 30605 tablet_bootstrap.cc:654] T 9acc12c9f0464e07965c05285fd334f0 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:04.685740 30606 raft_consensus.cc:348] T 9acc12c9f0464e07965c05285fd334f0 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:04.685830 30606 raft_consensus.cc:374] T 9acc12c9f0464e07965c05285fd334f0 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:04.685845 30604 tablet_bootstrap.cc:492] T 741da32f7c924490a5fa5933708adf30 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:04.685859 30606 raft_consensus.cc:729] T 9acc12c9f0464e07965c05285fd334f0 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:04.685957 30606 consensus_queue.cc:260] T 9acc12c9f0464e07965c05285fd334f0 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:04.686251 30606 ts_tablet_manager.cc:1398] T 9acc12c9f0464e07965c05285fd334f0 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:04.686625 30604 tablet_bootstrap.cc:654] T 741da32f7c924490a5fa5933708adf30 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:04.687278 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 741da32f7c924490a5fa5933708adf30 (DEFAULT_TABLE table=rebalance_test_table_9 [id=4b510e9cf32a4794a9193f7ebac46f93]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:04.687280 29164 tablet_service.cc:1467] Processing CreateTablet for tablet 3943d49acd0341c1a923d900281f1591 (DEFAULT_TABLE table=rebalance_test_table_9 [id=4b510e9cf32a4794a9193f7ebac46f93]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:04.687496 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 741da32f7c924490a5fa5933708adf30. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:04.687497 29164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3943d49acd0341c1a923d900281f1591. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:04.687590 30604 tablet_bootstrap.cc:492] T 741da32f7c924490a5fa5933708adf30 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:04.687673 30604 ts_tablet_manager.cc:1367] T 741da32f7c924490a5fa5933708adf30 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:04.688033 30604 raft_consensus.cc:348] T 741da32f7c924490a5fa5933708adf30 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:04.688119 30604 raft_consensus.cc:374] T 741da32f7c924490a5fa5933708adf30 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:04.688148 30604 raft_consensus.cc:729] T 741da32f7c924490a5fa5933708adf30 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:04.688256 30604 consensus_queue.cc:260] T 741da32f7c924490a5fa5933708adf30 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:04.688474 30604 ts_tablet_manager.cc:1398] T 741da32f7c924490a5fa5933708adf30 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:04.688616 30604 tablet_bootstrap.cc:492] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:04.689201 30606 tablet_bootstrap.cc:492] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:04.689370 29294 tablet_service.cc:1467] Processing CreateTablet for tablet 3943d49acd0341c1a923d900281f1591 (DEFAULT_TABLE table=rebalance_test_table_9 [id=4b510e9cf32a4794a9193f7ebac46f93]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:04.689508 30604 tablet_bootstrap.cc:654] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:04.689571 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3943d49acd0341c1a923d900281f1591. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:04.689913 30602 tablet_bootstrap.cc:492] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:04.690225 30606 tablet_bootstrap.cc:654] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:04.690709 30602 tablet_bootstrap.cc:654] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:04.690725 30604 tablet_bootstrap.cc:492] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:04.690795 30604 ts_tablet_manager.cc:1367] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:04.691188 30604 raft_consensus.cc:348] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:04.691278 30604 raft_consensus.cc:374] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:04.691308 30604 raft_consensus.cc:729] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:04.691414 30606 tablet_bootstrap.cc:492] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:04.691411 30604 consensus_queue.cc:260] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:04.691478 30606 ts_tablet_manager.cc:1367] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 20:46:04.691648 30604 ts_tablet_manager.cc:1398] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:04.691884 30606 raft_consensus.cc:348] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:04.691955 30606 raft_consensus.cc:374] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:04.691980 30606 raft_consensus.cc:729] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:04.692080 30606 consensus_queue.cc:260] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:04.692333 30606 ts_tablet_manager.cc:1398] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:04.692492 30606 tablet_bootstrap.cc:492] T 3943d49acd0341c1a923d900281f1591 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:04.693100 30606 tablet_bootstrap.cc:654] T 3943d49acd0341c1a923d900281f1591 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:04.693301 30605 tablet_bootstrap.cc:492] T 9acc12c9f0464e07965c05285fd334f0 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:04.693374 30605 ts_tablet_manager.cc:1367] T 9acc12c9f0464e07965c05285fd334f0 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20240502 20:46:04.693779 30605 raft_consensus.cc:348] T 9acc12c9f0464e07965c05285fd334f0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:04.693881 30605 raft_consensus.cc:374] T 9acc12c9f0464e07965c05285fd334f0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:04.693912 30605 raft_consensus.cc:729] T 9acc12c9f0464e07965c05285fd334f0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:04.694015 30605 consensus_queue.cc:260] T 9acc12c9f0464e07965c05285fd334f0 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:04.694293 30605 ts_tablet_manager.cc:1398] T 9acc12c9f0464e07965c05285fd334f0 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:04.694473 30605 tablet_bootstrap.cc:492] T 741da32f7c924490a5fa5933708adf30 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:04.695143 30606 tablet_bootstrap.cc:492] T 3943d49acd0341c1a923d900281f1591 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:04.695222 30606 ts_tablet_manager.cc:1367] T 3943d49acd0341c1a923d900281f1591 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 20:46:04.695241 30605 tablet_bootstrap.cc:654] T 741da32f7c924490a5fa5933708adf30 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:04.695539 30606 raft_consensus.cc:348] T 3943d49acd0341c1a923d900281f1591 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:04.695617 30606 raft_consensus.cc:374] T 3943d49acd0341c1a923d900281f1591 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:04.695645 30606 raft_consensus.cc:729] T 3943d49acd0341c1a923d900281f1591 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:04.695742 30606 consensus_queue.cc:260] T 3943d49acd0341c1a923d900281f1591 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:04.696095 30606 ts_tablet_manager.cc:1398] T 3943d49acd0341c1a923d900281f1591 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 20:46:04.697129 30605 tablet_bootstrap.cc:492] T 741da32f7c924490a5fa5933708adf30 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:04.697217 30605 ts_tablet_manager.cc:1367] T 741da32f7c924490a5fa5933708adf30 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:04.697316 30602 tablet_bootstrap.cc:492] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:04.697386 30602 ts_tablet_manager.cc:1367] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 20:46:04.697608 30605 raft_consensus.cc:348] T 741da32f7c924490a5fa5933708adf30 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:04.697706 30605 raft_consensus.cc:374] T 741da32f7c924490a5fa5933708adf30 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:04.697739 30605 raft_consensus.cc:729] T 741da32f7c924490a5fa5933708adf30 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:04.697796 30602 raft_consensus.cc:348] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:04.697847 30605 consensus_queue.cc:260] T 741da32f7c924490a5fa5933708adf30 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:04.697889 30602 raft_consensus.cc:374] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:04.697918 30602 raft_consensus.cc:729] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:04.698027 30602 consensus_queue.cc:260] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:04.698086 30605 ts_tablet_manager.cc:1398] T 741da32f7c924490a5fa5933708adf30 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:04.698230 30605 tablet_bootstrap.cc:492] T 3943d49acd0341c1a923d900281f1591 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:04.698283 30602 ts_tablet_manager.cc:1398] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:04.699064 30605 tablet_bootstrap.cc:654] T 3943d49acd0341c1a923d900281f1591 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:04.700637 30605 tablet_bootstrap.cc:492] T 3943d49acd0341c1a923d900281f1591 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:04.700711 30605 ts_tablet_manager.cc:1367] T 3943d49acd0341c1a923d900281f1591 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:04.701210 30605 raft_consensus.cc:348] T 3943d49acd0341c1a923d900281f1591 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:04.701301 30605 raft_consensus.cc:374] T 3943d49acd0341c1a923d900281f1591 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:04.701333 30605 raft_consensus.cc:729] T 3943d49acd0341c1a923d900281f1591 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:04.701428 30605 consensus_queue.cc:260] T 3943d49acd0341c1a923d900281f1591 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:04.701664 30605 ts_tablet_manager.cc:1398] T 3943d49acd0341c1a923d900281f1591 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:04.799109 30591 raft_consensus.cc:482] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:04.799217 30591 raft_consensus.cc:504] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:04.799566 30591 leader_election.cc:288] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:04.799907 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "741da32f7c924490a5fa5933708adf30" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:04.799921 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "741da32f7c924490a5fa5933708adf30" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:04.800041 29576 raft_consensus.cc:2454] T 741da32f7c924490a5fa5933708adf30 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:04.800047 29446 raft_consensus.cc:2454] T 741da32f7c924490a5fa5933708adf30 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:04.800307 29249 leader_election.cc:302] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:04.800463 30591 raft_consensus.cc:2781] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:04.800524 30591 raft_consensus.cc:482] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:04.800559 30591 raft_consensus.cc:3037] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:04.801345 30591 raft_consensus.cc:504] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:04.801640 30591 leader_election.cc:288] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:04.801924 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "741da32f7c924490a5fa5933708adf30" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:04.801937 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "741da32f7c924490a5fa5933708adf30" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:04.802021 29576 raft_consensus.cc:3037] T 741da32f7c924490a5fa5933708adf30 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:04.802026 29446 raft_consensus.cc:3037] T 741da32f7c924490a5fa5933708adf30 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:04.802829 29576 raft_consensus.cc:2454] T 741da32f7c924490a5fa5933708adf30 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:04.802834 29446 raft_consensus.cc:2454] T 741da32f7c924490a5fa5933708adf30 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:04.803074 29249 leader_election.cc:302] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:04.803236 30591 raft_consensus.cc:2781] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:04.803315 30591 raft_consensus.cc:686] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:04.803444 30591 consensus_queue.cc:237] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:04.805032 29038 catalog_manager.cc:5484] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:04.940735 30649 raft_consensus.cc:482] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:04.940845 30649 raft_consensus.cc:504] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:04.941174 30649 leader_election.cc:288] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:04.941560 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9acc12c9f0464e07965c05285fd334f0" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:04.941627 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9acc12c9f0464e07965c05285fd334f0" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:04.941722 29314 raft_consensus.cc:2454] T 9acc12c9f0464e07965c05285fd334f0 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:04.941747 29446 raft_consensus.cc:2454] T 9acc12c9f0464e07965c05285fd334f0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:04.942060 29512 leader_election.cc:302] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:04.942258 30649 raft_consensus.cc:2781] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:04.942322 30649 raft_consensus.cc:482] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:04.942348 30649 raft_consensus.cc:3037] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:04.943202 30649 raft_consensus.cc:504] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:04.943536 30649 leader_election.cc:288] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:04.943831 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9acc12c9f0464e07965c05285fd334f0" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:04.943940 29314 raft_consensus.cc:3037] T 9acc12c9f0464e07965c05285fd334f0 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:04.943930 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9acc12c9f0464e07965c05285fd334f0" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:04.944043 29446 raft_consensus.cc:3037] T 9acc12c9f0464e07965c05285fd334f0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:04.944690 29314 raft_consensus.cc:2454] T 9acc12c9f0464e07965c05285fd334f0 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:04.944917 29446 raft_consensus.cc:2454] T 9acc12c9f0464e07965c05285fd334f0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:04.944954 29512 leader_election.cc:302] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:04.945122 30649 raft_consensus.cc:2781] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:04.945288 30649 raft_consensus.cc:686] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Becoming Leader. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Running, Role: LEADER
I20240502 20:46:04.945417 30649 consensus_queue.cc:237] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:04.947064 29038 catalog_manager.cc:5484] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: term changed from 0 to 1, leader changed from <none> to db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:04.981457 30756 raft_consensus.cc:482] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:04.981564 30756 raft_consensus.cc:504] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:04.981940 30756 leader_election.cc:288] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:04.982275 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3943d49acd0341c1a923d900281f1591" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:04.982317 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3943d49acd0341c1a923d900281f1591" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:04.982395 29314 raft_consensus.cc:2454] T 3943d49acd0341c1a923d900281f1591 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:04.982440 29446 raft_consensus.cc:2454] T 3943d49acd0341c1a923d900281f1591 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:04.982685 29119 leader_election.cc:302] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:04.982860 30756 raft_consensus.cc:2781] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:04.982921 30756 raft_consensus.cc:482] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:04.982946 30756 raft_consensus.cc:3037] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:04.983865 30756 raft_consensus.cc:504] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:04.984194 30756 leader_election.cc:288] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:04.984513 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3943d49acd0341c1a923d900281f1591" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:04.984515 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3943d49acd0341c1a923d900281f1591" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:04.984601 29314 raft_consensus.cc:3037] T 3943d49acd0341c1a923d900281f1591 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:04.984607 29446 raft_consensus.cc:3037] T 3943d49acd0341c1a923d900281f1591 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:04.985318 29446 raft_consensus.cc:2454] T 3943d49acd0341c1a923d900281f1591 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:04.985323 29314 raft_consensus.cc:2454] T 3943d49acd0341c1a923d900281f1591 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:04.985555 29119 leader_election.cc:302] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:04.985797 30756 raft_consensus.cc:2781] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:04.985874 30756 raft_consensus.cc:686] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:04.986008 30756 consensus_queue.cc:237] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:04.988596 29038 catalog_manager.cc:5484] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
W20240502 20:46:05.156173 30774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:05.156371 30774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:05.159374 30774 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_9'
I20240502 20:46:05.205160 30591 consensus_queue.cc:1035] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:05.208958 30513 consensus_queue.cc:1035] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:05.247300 29314 raft_consensus.cc:1261] T 9acc12c9f0464e07965c05285fd334f0 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:05.247300 29446 raft_consensus.cc:1261] T 9acc12c9f0464e07965c05285fd334f0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:05.247687 30649 consensus_queue.cc:1035] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:05.248013 30771 consensus_queue.cc:1035] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:05.250113 29444 raft_consensus.cc:1261] T 3943d49acd0341c1a923d900281f1591 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:05.250475 30756 consensus_queue.cc:1035] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:05.251961 29314 raft_consensus.cc:1261] T 3943d49acd0341c1a923d900281f1591 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:05.252285 30756 consensus_queue.cc:1035] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 112.745 ms
time per row: 0.0563724 ms
I20240502 20:46:05.312192 29040 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalance_test_table_10"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:05.317320 29164 tablet_service.cc:1467] Processing CreateTablet for tablet 2c6dfe6a92284c5f96094eb16c757222 (DEFAULT_TABLE table=rebalance_test_table_10 [id=79c2e912ffdb4aae9cf78414231a842e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:05.317502 29294 tablet_service.cc:1467] Processing CreateTablet for tablet 2c6dfe6a92284c5f96094eb16c757222 (DEFAULT_TABLE table=rebalance_test_table_10 [id=79c2e912ffdb4aae9cf78414231a842e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:05.317610 29164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c6dfe6a92284c5f96094eb16c757222. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:05.317596 29425 tablet_service.cc:1467] Processing CreateTablet for tablet 88a8cf3a902d4e5bbbaf47a5202d514a (DEFAULT_TABLE table=rebalance_test_table_10 [id=79c2e912ffdb4aae9cf78414231a842e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:05.317720 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c6dfe6a92284c5f96094eb16c757222. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:05.317878 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88a8cf3a902d4e5bbbaf47a5202d514a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:05.318531 29163 tablet_service.cc:1467] Processing CreateTablet for tablet 88a8cf3a902d4e5bbbaf47a5202d514a (DEFAULT_TABLE table=rebalance_test_table_10 [id=79c2e912ffdb4aae9cf78414231a842e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:05.318758 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88a8cf3a902d4e5bbbaf47a5202d514a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:05.319972 30836 tablet_bootstrap.cc:492] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:05.320726 30836 tablet_bootstrap.cc:654] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:05.320672 29162 tablet_service.cc:1467] Processing CreateTablet for tablet 9b609e0004524a67bb4eea7b3c5d58fb (DEFAULT_TABLE table=rebalance_test_table_10 [id=79c2e912ffdb4aae9cf78414231a842e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:05.320940 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b609e0004524a67bb4eea7b3c5d58fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:05.321738 30836 tablet_bootstrap.cc:492] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:05.321801 30836 ts_tablet_manager.cc:1367] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:05.322150 30836 raft_consensus.cc:348] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:05.322227 30836 raft_consensus.cc:374] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:05.322245 30836 raft_consensus.cc:729] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:05.322345 30836 consensus_queue.cc:260] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:05.322605 30836 ts_tablet_manager.cc:1398] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:05.323015 29294 tablet_service.cc:1467] Processing CreateTablet for tablet 9b609e0004524a67bb4eea7b3c5d58fb (DEFAULT_TABLE table=rebalance_test_table_10 [id=79c2e912ffdb4aae9cf78414231a842e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:05.323220 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b609e0004524a67bb4eea7b3c5d58fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:05.323458 30835 tablet_bootstrap.cc:492] T 2c6dfe6a92284c5f96094eb16c757222 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:05.324277 30835 tablet_bootstrap.cc:654] T 2c6dfe6a92284c5f96094eb16c757222 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:05.324419 29556 tablet_service.cc:1467] Processing CreateTablet for tablet 2c6dfe6a92284c5f96094eb16c757222 (DEFAULT_TABLE table=rebalance_test_table_10 [id=79c2e912ffdb4aae9cf78414231a842e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:05.324652 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c6dfe6a92284c5f96094eb16c757222. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:05.324803 30834 tablet_bootstrap.cc:492] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:05.325621 30834 tablet_bootstrap.cc:654] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:05.326864 30835 tablet_bootstrap.cc:492] T 2c6dfe6a92284c5f96094eb16c757222 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:05.326954 30835 ts_tablet_manager.cc:1367] T 2c6dfe6a92284c5f96094eb16c757222 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:05.326917 29555 tablet_service.cc:1467] Processing CreateTablet for tablet 88a8cf3a902d4e5bbbaf47a5202d514a (DEFAULT_TABLE table=rebalance_test_table_10 [id=79c2e912ffdb4aae9cf78414231a842e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:05.327118 29555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88a8cf3a902d4e5bbbaf47a5202d514a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:05.327347 30835 raft_consensus.cc:348] T 2c6dfe6a92284c5f96094eb16c757222 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:05.327453 30835 raft_consensus.cc:374] T 2c6dfe6a92284c5f96094eb16c757222 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:05.327489 30835 raft_consensus.cc:729] T 2c6dfe6a92284c5f96094eb16c757222 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:05.327602 30835 consensus_queue.cc:260] T 2c6dfe6a92284c5f96094eb16c757222 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:05.327870 30835 ts_tablet_manager.cc:1398] T 2c6dfe6a92284c5f96094eb16c757222 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:05.328256 30834 tablet_bootstrap.cc:492] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:05.328342 30834 ts_tablet_manager.cc:1367] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:05.328768 30834 raft_consensus.cc:348] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:05.328871 30834 raft_consensus.cc:374] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:05.328912 30834 raft_consensus.cc:729] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:05.328992 29557 tablet_service.cc:1467] Processing CreateTablet for tablet 9b609e0004524a67bb4eea7b3c5d58fb (DEFAULT_TABLE table=rebalance_test_table_10 [id=79c2e912ffdb4aae9cf78414231a842e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:05.329030 30834 consensus_queue.cc:260] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:05.329209 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b609e0004524a67bb4eea7b3c5d58fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:05.329316 30834 ts_tablet_manager.cc:1398] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:05.329478 30834 tablet_bootstrap.cc:492] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:05.330255 30834 tablet_bootstrap.cc:654] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:05.330945 30835 tablet_bootstrap.cc:492] T 88a8cf3a902d4e5bbbaf47a5202d514a P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:05.331643 30835 tablet_bootstrap.cc:654] T 88a8cf3a902d4e5bbbaf47a5202d514a P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:05.332373 30834 tablet_bootstrap.cc:492] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:05.332459 30834 ts_tablet_manager.cc:1367] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 20:46:05.333065 30834 raft_consensus.cc:348] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:05.333146 30834 raft_consensus.cc:374] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:05.333169 30834 raft_consensus.cc:729] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:05.333248 30834 consensus_queue.cc:260] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:05.333485 30834 ts_tablet_manager.cc:1398] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:05.338541 30840 tablet_bootstrap.cc:492] T 2c6dfe6a92284c5f96094eb16c757222 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:05.339239 30840 tablet_bootstrap.cc:654] T 2c6dfe6a92284c5f96094eb16c757222 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:05.340180 30840 tablet_bootstrap.cc:492] T 2c6dfe6a92284c5f96094eb16c757222 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:05.340260 30840 ts_tablet_manager.cc:1367] T 2c6dfe6a92284c5f96094eb16c757222 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240502 20:46:05.340626 30840 raft_consensus.cc:348] T 2c6dfe6a92284c5f96094eb16c757222 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:05.340730 30840 raft_consensus.cc:374] T 2c6dfe6a92284c5f96094eb16c757222 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:05.340762 30840 raft_consensus.cc:729] T 2c6dfe6a92284c5f96094eb16c757222 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:05.340863 30840 consensus_queue.cc:260] T 2c6dfe6a92284c5f96094eb16c757222 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:05.341094 30840 ts_tablet_manager.cc:1398] T 2c6dfe6a92284c5f96094eb16c757222 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:05.341259 30840 tablet_bootstrap.cc:492] T 88a8cf3a902d4e5bbbaf47a5202d514a P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:05.342191 30840 tablet_bootstrap.cc:654] T 88a8cf3a902d4e5bbbaf47a5202d514a P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:05.342978 30835 tablet_bootstrap.cc:492] T 88a8cf3a902d4e5bbbaf47a5202d514a P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:05.343050 30835 ts_tablet_manager.cc:1367] T 88a8cf3a902d4e5bbbaf47a5202d514a P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20240502 20:46:05.343398 30840 tablet_bootstrap.cc:492] T 88a8cf3a902d4e5bbbaf47a5202d514a P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:05.343392 30835 raft_consensus.cc:348] T 88a8cf3a902d4e5bbbaf47a5202d514a P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:05.343484 30835 raft_consensus.cc:374] T 88a8cf3a902d4e5bbbaf47a5202d514a P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:05.343515 30835 raft_consensus.cc:729] T 88a8cf3a902d4e5bbbaf47a5202d514a P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:05.343482 30840 ts_tablet_manager.cc:1367] T 88a8cf3a902d4e5bbbaf47a5202d514a P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 20:46:05.343626 30835 consensus_queue.cc:260] T 88a8cf3a902d4e5bbbaf47a5202d514a P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:05.343940 30840 raft_consensus.cc:348] T 88a8cf3a902d4e5bbbaf47a5202d514a P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:05.344053 30840 raft_consensus.cc:374] T 88a8cf3a902d4e5bbbaf47a5202d514a P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:05.344085 30840 raft_consensus.cc:729] T 88a8cf3a902d4e5bbbaf47a5202d514a P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:05.344280 30835 ts_tablet_manager.cc:1398] T 88a8cf3a902d4e5bbbaf47a5202d514a P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:05.344424 30835 tablet_bootstrap.cc:492] T 9b609e0004524a67bb4eea7b3c5d58fb P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:05.345050 30835 tablet_bootstrap.cc:654] T 9b609e0004524a67bb4eea7b3c5d58fb P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:05.345171 30840 consensus_queue.cc:260] T 88a8cf3a902d4e5bbbaf47a5202d514a P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:05.345796 30840 ts_tablet_manager.cc:1398] T 88a8cf3a902d4e5bbbaf47a5202d514a P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:05.345959 30840 tablet_bootstrap.cc:492] T 9b609e0004524a67bb4eea7b3c5d58fb P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:05.347115 30840 tablet_bootstrap.cc:654] T 9b609e0004524a67bb4eea7b3c5d58fb P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:05.348201 30840 tablet_bootstrap.cc:492] T 9b609e0004524a67bb4eea7b3c5d58fb P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:05.348289 30840 ts_tablet_manager.cc:1367] T 9b609e0004524a67bb4eea7b3c5d58fb P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 20:46:05.348671 30840 raft_consensus.cc:348] T 9b609e0004524a67bb4eea7b3c5d58fb P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:05.348770 30840 raft_consensus.cc:374] T 9b609e0004524a67bb4eea7b3c5d58fb P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:05.348814 30840 raft_consensus.cc:729] T 9b609e0004524a67bb4eea7b3c5d58fb P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:05.348924 30840 consensus_queue.cc:260] T 9b609e0004524a67bb4eea7b3c5d58fb P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:05.349184 30840 ts_tablet_manager.cc:1398] T 9b609e0004524a67bb4eea7b3c5d58fb P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:05.355028 30835 tablet_bootstrap.cc:492] T 9b609e0004524a67bb4eea7b3c5d58fb P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:05.355113 30835 ts_tablet_manager.cc:1367] T 9b609e0004524a67bb4eea7b3c5d58fb P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240502 20:46:05.355445 30835 raft_consensus.cc:348] T 9b609e0004524a67bb4eea7b3c5d58fb P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:05.355547 30835 raft_consensus.cc:374] T 9b609e0004524a67bb4eea7b3c5d58fb P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:05.355587 30835 raft_consensus.cc:729] T 9b609e0004524a67bb4eea7b3c5d58fb P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:05.355705 30835 consensus_queue.cc:260] T 9b609e0004524a67bb4eea7b3c5d58fb P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:05.355939 30835 ts_tablet_manager.cc:1398] T 9b609e0004524a67bb4eea7b3c5d58fb P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:05.431135 30513 raft_consensus.cc:482] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:05.431243 30513 raft_consensus.cc:504] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:05.431587 30513 leader_election.cc:288] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:05.431905 29184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c6dfe6a92284c5f96094eb16c757222" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:05.431912 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c6dfe6a92284c5f96094eb16c757222" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:05.432030 29184 raft_consensus.cc:2454] T 2c6dfe6a92284c5f96094eb16c757222 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:05.432029 29576 raft_consensus.cc:2454] T 2c6dfe6a92284c5f96094eb16c757222 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:05.432319 29249 leader_election.cc:302] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:05.432487 30513 raft_consensus.cc:2781] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:05.432556 30513 raft_consensus.cc:482] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:05.432600 30513 raft_consensus.cc:3037] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:05.433369 30513 raft_consensus.cc:504] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:05.433650 30513 leader_election.cc:288] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:05.433916 29184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c6dfe6a92284c5f96094eb16c757222" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:05.433931 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c6dfe6a92284c5f96094eb16c757222" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:05.434007 29184 raft_consensus.cc:3037] T 2c6dfe6a92284c5f96094eb16c757222 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:05.434031 29576 raft_consensus.cc:3037] T 2c6dfe6a92284c5f96094eb16c757222 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:05.434753 29184 raft_consensus.cc:2454] T 2c6dfe6a92284c5f96094eb16c757222 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:05.434808 29576 raft_consensus.cc:2454] T 2c6dfe6a92284c5f96094eb16c757222 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:05.435019 29251 leader_election.cc:302] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:05.435185 30513 raft_consensus.cc:2781] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:05.435266 30513 raft_consensus.cc:686] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:05.435410 30513 consensus_queue.cc:237] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:05.437085 29038 catalog_manager.cc:5484] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:05.532855 30659 raft_consensus.cc:482] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:05.532966 30659 raft_consensus.cc:504] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:05.533293 30659 leader_election.cc:288] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:05.533656 29184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88a8cf3a902d4e5bbbaf47a5202d514a" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:05.533646 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88a8cf3a902d4e5bbbaf47a5202d514a" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:05.533808 29576 raft_consensus.cc:2454] T 88a8cf3a902d4e5bbbaf47a5202d514a P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:05.533808 29184 raft_consensus.cc:2454] T 88a8cf3a902d4e5bbbaf47a5202d514a P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:05.534081 29382 leader_election.cc:302] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:05.534261 30659 raft_consensus.cc:2781] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:05.534319 30659 raft_consensus.cc:482] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:05.534344 30659 raft_consensus.cc:3037] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:05.535172 30659 raft_consensus.cc:504] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:05.535470 30659 leader_election.cc:288] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:05.535771 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88a8cf3a902d4e5bbbaf47a5202d514a" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:05.535755 29184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88a8cf3a902d4e5bbbaf47a5202d514a" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:05.535876 29576 raft_consensus.cc:3037] T 88a8cf3a902d4e5bbbaf47a5202d514a P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:05.535888 29184 raft_consensus.cc:3037] T 88a8cf3a902d4e5bbbaf47a5202d514a P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:05.536558 29576 raft_consensus.cc:2454] T 88a8cf3a902d4e5bbbaf47a5202d514a P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:05.536644 29184 raft_consensus.cc:2454] T 88a8cf3a902d4e5bbbaf47a5202d514a P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:05.536818 29380 leader_election.cc:302] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:05.536957 30659 raft_consensus.cc:2781] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:05.537143 30659 raft_consensus.cc:686] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:05.537279 30659 consensus_queue.cc:237] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:05.539182 29038 catalog_manager.cc:5484] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:05.623776 30591 raft_consensus.cc:482] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:05.623894 30591 raft_consensus.cc:504] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:05.624286 30591 leader_election.cc:288] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:05.624611 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b609e0004524a67bb4eea7b3c5d58fb" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:05.624622 29184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b609e0004524a67bb4eea7b3c5d58fb" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:05.624732 29184 raft_consensus.cc:2454] T 9b609e0004524a67bb4eea7b3c5d58fb P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:05.624745 29576 raft_consensus.cc:2454] T 9b609e0004524a67bb4eea7b3c5d58fb P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:05.625005 29251 leader_election.cc:302] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:05.625165 30591 raft_consensus.cc:2781] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:05.625224 30591 raft_consensus.cc:482] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:05.625262 30591 raft_consensus.cc:3037] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:05.626261 30591 raft_consensus.cc:504] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:05.626629 30591 leader_election.cc:288] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:05.626962 29184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b609e0004524a67bb4eea7b3c5d58fb" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:05.627074 29184 raft_consensus.cc:3037] T 9b609e0004524a67bb4eea7b3c5d58fb P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:05.627771 29184 raft_consensus.cc:2454] T 9b609e0004524a67bb4eea7b3c5d58fb P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:05.627979 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b609e0004524a67bb4eea7b3c5d58fb" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:05.628024 29251 leader_election.cc:302] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:05.628098 29576 raft_consensus.cc:3037] T 9b609e0004524a67bb4eea7b3c5d58fb P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:05.628154 30591 raft_consensus.cc:2781] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:05.628291 30591 raft_consensus.cc:686] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:05.628391 30591 consensus_queue.cc:237] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:05.628896 29576 raft_consensus.cc:2454] T 9b609e0004524a67bb4eea7b3c5d58fb P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:05.630093 29038 catalog_manager.cc:5484] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
W20240502 20:46:05.788623 30851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:05.788829 30851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:05.791882 30851 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_10'
I20240502 20:46:05.869339 29184 raft_consensus.cc:1261] T 9b609e0004524a67bb4eea7b3c5d58fb P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:05.869807 29576 raft_consensus.cc:1261] T 2c6dfe6a92284c5f96094eb16c757222 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:05.869771 30591 consensus_queue.cc:1035] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:05.870198 30591 consensus_queue.cc:1035] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:05.871433 29182 raft_consensus.cc:1261] T 2c6dfe6a92284c5f96094eb16c757222 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:05.871870 30591 consensus_queue.cc:1035] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:05.872735 29575 raft_consensus.cc:1261] T 9b609e0004524a67bb4eea7b3c5d58fb P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:05.873117 30591 consensus_queue.cc:1035] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:05.875151 29577 raft_consensus.cc:1261] T 88a8cf3a902d4e5bbbaf47a5202d514a P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:05.875224 29175 raft_consensus.cc:1261] T 88a8cf3a902d4e5bbbaf47a5202d514a P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:05.875510 30659 consensus_queue.cc:1035] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:05.875854 30659 consensus_queue.cc:1035] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 94.0459 ms
time per row: 0.047023 ms
I20240502 20:46:05.928478 29040 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalance_test_table_11"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:05.933463 29294 tablet_service.cc:1467] Processing CreateTablet for tablet d167293496fb4334ae670a62a2f80fa9 (DEFAULT_TABLE table=rebalance_test_table_11 [id=cc807ad977d146dc84be7ee2897657fc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:05.933619 29425 tablet_service.cc:1467] Processing CreateTablet for tablet d167293496fb4334ae670a62a2f80fa9 (DEFAULT_TABLE table=rebalance_test_table_11 [id=cc807ad977d146dc84be7ee2897657fc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:05.933771 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d167293496fb4334ae670a62a2f80fa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:05.933897 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d167293496fb4334ae670a62a2f80fa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:05.933897 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 3c3bdc0b9e934ae2b81317a371817563 (DEFAULT_TABLE table=rebalance_test_table_11 [id=cc807ad977d146dc84be7ee2897657fc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:05.934118 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c3bdc0b9e934ae2b81317a371817563. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:05.934098 29162 tablet_service.cc:1467] Processing CreateTablet for tablet d167293496fb4334ae670a62a2f80fa9 (DEFAULT_TABLE table=rebalance_test_table_11 [id=cc807ad977d146dc84be7ee2897657fc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:05.934334 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d167293496fb4334ae670a62a2f80fa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:05.934829 29424 tablet_service.cc:1467] Processing CreateTablet for tablet 3c3bdc0b9e934ae2b81317a371817563 (DEFAULT_TABLE table=rebalance_test_table_11 [id=cc807ad977d146dc84be7ee2897657fc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:05.935024 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c3bdc0b9e934ae2b81317a371817563. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:05.936407 29555 tablet_service.cc:1467] Processing CreateTablet for tablet a0e1f5ce5c544045b556cb31e744b169 (DEFAULT_TABLE table=rebalance_test_table_11 [id=cc807ad977d146dc84be7ee2897657fc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:05.936687 29555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0e1f5ce5c544045b556cb31e744b169. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:05.936407 29557 tablet_service.cc:1467] Processing CreateTablet for tablet 3c3bdc0b9e934ae2b81317a371817563 (DEFAULT_TABLE table=rebalance_test_table_11 [id=cc807ad977d146dc84be7ee2897657fc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:05.936874 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c3bdc0b9e934ae2b81317a371817563. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:05.937407 29162 tablet_service.cc:1467] Processing CreateTablet for tablet a0e1f5ce5c544045b556cb31e744b169 (DEFAULT_TABLE table=rebalance_test_table_11 [id=cc807ad977d146dc84be7ee2897657fc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:05.937613 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0e1f5ce5c544045b556cb31e744b169. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:05.939049 30911 tablet_bootstrap.cc:492] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:05.939615 30910 tablet_bootstrap.cc:492] T d167293496fb4334ae670a62a2f80fa9 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:05.939630 29424 tablet_service.cc:1467] Processing CreateTablet for tablet a0e1f5ce5c544045b556cb31e744b169 (DEFAULT_TABLE table=rebalance_test_table_11 [id=cc807ad977d146dc84be7ee2897657fc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:05.939826 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0e1f5ce5c544045b556cb31e744b169. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:05.939951 30911 tablet_bootstrap.cc:654] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:05.940784 30910 tablet_bootstrap.cc:654] T d167293496fb4334ae670a62a2f80fa9 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:05.941105 30911 tablet_bootstrap.cc:492] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:05.941190 30911 ts_tablet_manager.cc:1367] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:05.941355 30912 tablet_bootstrap.cc:492] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:05.941614 30911 raft_consensus.cc:348] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:05.941718 30911 raft_consensus.cc:374] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:05.941730 30913 tablet_bootstrap.cc:492] T a0e1f5ce5c544045b556cb31e744b169 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:05.941751 30911 raft_consensus.cc:729] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:05.941870 30911 consensus_queue.cc:260] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:05.942109 30911 ts_tablet_manager.cc:1398] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:05.942111 30912 tablet_bootstrap.cc:654] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:05.942267 30911 tablet_bootstrap.cc:492] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:05.942526 30913 tablet_bootstrap.cc:654] T a0e1f5ce5c544045b556cb31e744b169 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:05.943583 30910 tablet_bootstrap.cc:492] T d167293496fb4334ae670a62a2f80fa9 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:05.943663 30910 ts_tablet_manager.cc:1367] T d167293496fb4334ae670a62a2f80fa9 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:05.943842 30911 tablet_bootstrap.cc:654] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:05.944470 30910 raft_consensus.cc:348] T d167293496fb4334ae670a62a2f80fa9 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:05.944568 30910 raft_consensus.cc:374] T d167293496fb4334ae670a62a2f80fa9 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:05.944597 30910 raft_consensus.cc:729] T d167293496fb4334ae670a62a2f80fa9 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:05.944702 30910 consensus_queue.cc:260] T d167293496fb4334ae670a62a2f80fa9 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:05.944840 30911 tablet_bootstrap.cc:492] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:05.944924 30911 ts_tablet_manager.cc:1367] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:05.944983 30910 ts_tablet_manager.cc:1398] T d167293496fb4334ae670a62a2f80fa9 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:05.945137 30910 tablet_bootstrap.cc:492] T 3c3bdc0b9e934ae2b81317a371817563 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:05.945250 30911 raft_consensus.cc:348] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:05.945360 30911 raft_consensus.cc:374] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:05.945398 30911 raft_consensus.cc:729] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:05.945513 30911 consensus_queue.cc:260] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:05.945762 30911 ts_tablet_manager.cc:1398] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:05.945989 30910 tablet_bootstrap.cc:654] T 3c3bdc0b9e934ae2b81317a371817563 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:05.947402 30912 tablet_bootstrap.cc:492] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:05.947445 30913 tablet_bootstrap.cc:492] T a0e1f5ce5c544045b556cb31e744b169 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:05.947536 30912 ts_tablet_manager.cc:1367] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 20:46:05.947571 30913 ts_tablet_manager.cc:1367] T a0e1f5ce5c544045b556cb31e744b169 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20240502 20:46:05.947974 30912 raft_consensus.cc:348] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:05.948082 30912 raft_consensus.cc:374] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:05.948114 30912 raft_consensus.cc:729] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:05.948155 30913 raft_consensus.cc:348] T a0e1f5ce5c544045b556cb31e744b169 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:05.948251 30913 raft_consensus.cc:374] T a0e1f5ce5c544045b556cb31e744b169 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:05.948253 30912 consensus_queue.cc:260] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:05.948352 30913 raft_consensus.cc:729] T a0e1f5ce5c544045b556cb31e744b169 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:05.948527 30912 ts_tablet_manager.cc:1398] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:05.948516 30913 consensus_queue.cc:260] T a0e1f5ce5c544045b556cb31e744b169 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:05.948689 30912 tablet_bootstrap.cc:492] T d167293496fb4334ae670a62a2f80fa9 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:05.948992 30913 ts_tablet_manager.cc:1398] T a0e1f5ce5c544045b556cb31e744b169 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:05.949188 30913 tablet_bootstrap.cc:492] T 3c3bdc0b9e934ae2b81317a371817563 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:05.950055 30913 tablet_bootstrap.cc:654] T 3c3bdc0b9e934ae2b81317a371817563 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:05.950282 30912 tablet_bootstrap.cc:654] T d167293496fb4334ae670a62a2f80fa9 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:05.951388 30910 tablet_bootstrap.cc:492] T 3c3bdc0b9e934ae2b81317a371817563 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:05.951514 30913 tablet_bootstrap.cc:492] T 3c3bdc0b9e934ae2b81317a371817563 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:05.951514 30910 ts_tablet_manager.cc:1367] T 3c3bdc0b9e934ae2b81317a371817563 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 20:46:05.951586 30913 ts_tablet_manager.cc:1367] T 3c3bdc0b9e934ae2b81317a371817563 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:05.952009 30910 raft_consensus.cc:348] T 3c3bdc0b9e934ae2b81317a371817563 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:05.952004 30913 raft_consensus.cc:348] T 3c3bdc0b9e934ae2b81317a371817563 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:05.952097 30910 raft_consensus.cc:374] T 3c3bdc0b9e934ae2b81317a371817563 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:05.952097 30913 raft_consensus.cc:374] T 3c3bdc0b9e934ae2b81317a371817563 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:05.952127 30913 raft_consensus.cc:729] T 3c3bdc0b9e934ae2b81317a371817563 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:05.952127 30910 raft_consensus.cc:729] T 3c3bdc0b9e934ae2b81317a371817563 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:05.952239 30910 consensus_queue.cc:260] T 3c3bdc0b9e934ae2b81317a371817563 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:05.952414 30913 consensus_queue.cc:260] T 3c3bdc0b9e934ae2b81317a371817563 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:05.952603 30910 ts_tablet_manager.cc:1398] T 3c3bdc0b9e934ae2b81317a371817563 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:05.952662 30913 ts_tablet_manager.cc:1398] T 3c3bdc0b9e934ae2b81317a371817563 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:05.953647 30912 tablet_bootstrap.cc:492] T d167293496fb4334ae670a62a2f80fa9 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:05.953735 30912 ts_tablet_manager.cc:1367] T d167293496fb4334ae670a62a2f80fa9 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 20:46:05.954140 30912 raft_consensus.cc:348] T d167293496fb4334ae670a62a2f80fa9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:05.954233 30912 raft_consensus.cc:374] T d167293496fb4334ae670a62a2f80fa9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:05.954272 30912 raft_consensus.cc:729] T d167293496fb4334ae670a62a2f80fa9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:05.954386 30912 consensus_queue.cc:260] T d167293496fb4334ae670a62a2f80fa9 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:05.954737 30912 ts_tablet_manager.cc:1398] T d167293496fb4334ae670a62a2f80fa9 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:05.954916 30912 tablet_bootstrap.cc:492] T a0e1f5ce5c544045b556cb31e744b169 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:05.955683 30912 tablet_bootstrap.cc:654] T a0e1f5ce5c544045b556cb31e744b169 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:05.956825 30912 tablet_bootstrap.cc:492] T a0e1f5ce5c544045b556cb31e744b169 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:05.956907 30912 ts_tablet_manager.cc:1367] T a0e1f5ce5c544045b556cb31e744b169 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:05.957413 30912 raft_consensus.cc:348] T a0e1f5ce5c544045b556cb31e744b169 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:05.957518 30912 raft_consensus.cc:374] T a0e1f5ce5c544045b556cb31e744b169 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:05.957561 30912 raft_consensus.cc:729] T a0e1f5ce5c544045b556cb31e744b169 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:05.957675 30912 consensus_queue.cc:260] T a0e1f5ce5c544045b556cb31e744b169 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:05.957960 30912 ts_tablet_manager.cc:1398] T a0e1f5ce5c544045b556cb31e744b169 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:05.979808 30916 raft_consensus.cc:482] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:05.979897 30916 raft_consensus.cc:504] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:05.980201 30916 leader_election.cc:288] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:05.980495 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d167293496fb4334ae670a62a2f80fa9" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:05.980547 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d167293496fb4334ae670a62a2f80fa9" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:05.980612 29314 raft_consensus.cc:2454] T d167293496fb4334ae670a62a2f80fa9 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:05.980659 29444 raft_consensus.cc:2454] T d167293496fb4334ae670a62a2f80fa9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:05.980875 29119 leader_election.cc:302] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:05.981045 30916 raft_consensus.cc:2781] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:05.981109 30916 raft_consensus.cc:482] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:05.981139 30916 raft_consensus.cc:3037] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:05.981887 30916 raft_consensus.cc:504] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:05.982169 30916 leader_election.cc:288] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:05.982445 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d167293496fb4334ae670a62a2f80fa9" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:05.982520 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d167293496fb4334ae670a62a2f80fa9" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:05.982569 29314 raft_consensus.cc:3037] T d167293496fb4334ae670a62a2f80fa9 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:05.982600 29444 raft_consensus.cc:3037] T d167293496fb4334ae670a62a2f80fa9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:05.983243 29444 raft_consensus.cc:2454] T d167293496fb4334ae670a62a2f80fa9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:05.983246 29314 raft_consensus.cc:2454] T d167293496fb4334ae670a62a2f80fa9 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:05.983464 29119 leader_election.cc:302] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:05.983610 30916 raft_consensus.cc:2781] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:05.983696 30916 raft_consensus.cc:686] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:05.983829 30916 consensus_queue.cc:237] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:05.985375 29033 catalog_manager.cc:5484] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:06.038360 30916 raft_consensus.cc:482] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:06.038484 30916 raft_consensus.cc:504] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:06.038805 30916 leader_election.cc:288] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:06.039113 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0e1f5ce5c544045b556cb31e744b169" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:06.039116 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0e1f5ce5c544045b556cb31e744b169" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:06.039252 29444 raft_consensus.cc:2454] T a0e1f5ce5c544045b556cb31e744b169 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:06.039252 29575 raft_consensus.cc:2454] T a0e1f5ce5c544045b556cb31e744b169 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:06.039525 29118 leader_election.cc:302] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:06.040504 30916 raft_consensus.cc:2781] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:06.040565 30916 raft_consensus.cc:482] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:06.040603 30916 raft_consensus.cc:3037] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:06.041321 30916 raft_consensus.cc:504] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:06.041620 30916 leader_election.cc:288] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:06.041891 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0e1f5ce5c544045b556cb31e744b169" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:06.041916 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0e1f5ce5c544045b556cb31e744b169" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:06.042006 29444 raft_consensus.cc:3037] T a0e1f5ce5c544045b556cb31e744b169 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:06.042006 29575 raft_consensus.cc:3037] T a0e1f5ce5c544045b556cb31e744b169 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:06.042896 29444 raft_consensus.cc:2454] T a0e1f5ce5c544045b556cb31e744b169 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:06.042896 29575 raft_consensus.cc:2454] T a0e1f5ce5c544045b556cb31e744b169 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:06.043149 29119 leader_election.cc:302] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:06.043321 30916 raft_consensus.cc:2781] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:06.043413 30916 raft_consensus.cc:686] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:06.043543 30916 consensus_queue.cc:237] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:06.045260 29033 catalog_manager.cc:5484] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:06.115154 30892 raft_consensus.cc:482] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:06.115268 30892 raft_consensus.cc:504] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:06.115579 30892 leader_election.cc:288] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:06.115883 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c3bdc0b9e934ae2b81317a371817563" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:06.115887 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c3bdc0b9e934ae2b81317a371817563" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:06.116019 29315 raft_consensus.cc:2454] T 3c3bdc0b9e934ae2b81317a371817563 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:06.116019 29575 raft_consensus.cc:2454] T 3c3bdc0b9e934ae2b81317a371817563 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:06.116266 29380 leader_election.cc:302] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:06.116420 30892 raft_consensus.cc:2781] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:06.116480 30892 raft_consensus.cc:482] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:06.116525 30892 raft_consensus.cc:3037] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:06.117271 30892 raft_consensus.cc:504] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:06.117560 30892 leader_election.cc:288] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:06.117853 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c3bdc0b9e934ae2b81317a371817563" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:06.117869 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c3bdc0b9e934ae2b81317a371817563" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:06.117977 29575 raft_consensus.cc:3037] T 3c3bdc0b9e934ae2b81317a371817563 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:06.117977 29315 raft_consensus.cc:3037] T 3c3bdc0b9e934ae2b81317a371817563 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:06.118886 29315 raft_consensus.cc:2454] T 3c3bdc0b9e934ae2b81317a371817563 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:06.118886 29575 raft_consensus.cc:2454] T 3c3bdc0b9e934ae2b81317a371817563 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:06.119164 29380 leader_election.cc:302] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:06.119311 30892 raft_consensus.cc:2781] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:06.119416 30892 raft_consensus.cc:686] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:06.119540 30892 consensus_queue.cc:237] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:06.121207 29033 catalog_manager.cc:5484] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
W20240502 20:46:06.236889 30927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:06.237123 30927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:06.240120 30927 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_11'
I20240502 20:46:06.325208 29315 raft_consensus.cc:1261] T d167293496fb4334ae670a62a2f80fa9 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:06.325495 29444 raft_consensus.cc:1261] T d167293496fb4334ae670a62a2f80fa9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:06.325624 30756 consensus_queue.cc:1035] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:06.325660 29575 raft_consensus.cc:1261] T a0e1f5ce5c544045b556cb31e744b169 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:06.325870 29446 raft_consensus.cc:1261] T a0e1f5ce5c544045b556cb31e744b169 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:06.326144 30756 consensus_queue.cc:1035] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:06.326745 30916 consensus_queue.cc:1035] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:06.327032 30916 consensus_queue.cc:1035] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:06.328053 29314 raft_consensus.cc:1261] T 3c3bdc0b9e934ae2b81317a371817563 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:06.330659 29576 raft_consensus.cc:1261] T 3c3bdc0b9e934ae2b81317a371817563 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:06.331523 30892 consensus_queue.cc:1035] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:06.331751 30893 consensus_queue.cc:1035] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:06.351032 30895 mvcc.cc:205] Tried to move back new op lower bound from 7023340610875428864 to 7023340610026782720. Current Snapshot: MvccSnapshot[applied={T|T < 7023340610875428864}]
I20240502 20:46:06.375393 30883 mvcc.cc:205] Tried to move back new op lower bound from 7023340610861682688 to 7023340609470816256. Current Snapshot: MvccSnapshot[applied={T|T < 7023340610861682688}]
INSERT report
rows total: 2000
time total: 120.422 ms
time per row: 0.060211 ms
I20240502 20:46:06.402176 29039 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalance_test_table_12"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:06.407498 29294 tablet_service.cc:1467] Processing CreateTablet for tablet b0e8b1909df34fce92f94d6af5f5c3b6 (DEFAULT_TABLE table=rebalance_test_table_12 [id=e2320262ebe84e399b45c01bf9a7d73d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:06.407581 29424 tablet_service.cc:1467] Processing CreateTablet for tablet b0e8b1909df34fce92f94d6af5f5c3b6 (DEFAULT_TABLE table=rebalance_test_table_12 [id=e2320262ebe84e399b45c01bf9a7d73d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:06.407722 29557 tablet_service.cc:1467] Processing CreateTablet for tablet f65eb8cde25e483b9c790740ccfd357c (DEFAULT_TABLE table=rebalance_test_table_12 [id=e2320262ebe84e399b45c01bf9a7d73d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:06.407817 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0e8b1909df34fce92f94d6af5f5c3b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:06.407891 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0e8b1909df34fce92f94d6af5f5c3b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:06.407963 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f65eb8cde25e483b9c790740ccfd357c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:06.408102 29162 tablet_service.cc:1467] Processing CreateTablet for tablet b0e8b1909df34fce92f94d6af5f5c3b6 (DEFAULT_TABLE table=rebalance_test_table_12 [id=e2320262ebe84e399b45c01bf9a7d73d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:06.408361 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0e8b1909df34fce92f94d6af5f5c3b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:06.410193 30913 tablet_bootstrap.cc:492] T f65eb8cde25e483b9c790740ccfd357c P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:06.410370 30911 tablet_bootstrap.cc:492] T b0e8b1909df34fce92f94d6af5f5c3b6 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:06.410573 30910 tablet_bootstrap.cc:492] T b0e8b1909df34fce92f94d6af5f5c3b6 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:06.411248 30910 tablet_bootstrap.cc:654] T b0e8b1909df34fce92f94d6af5f5c3b6 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:06.411262 30911 tablet_bootstrap.cc:654] T b0e8b1909df34fce92f94d6af5f5c3b6 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:06.411262 30913 tablet_bootstrap.cc:654] T f65eb8cde25e483b9c790740ccfd357c P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:06.410444 30912 tablet_bootstrap.cc:492] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:06.412918 30912 tablet_bootstrap.cc:654] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:06.413141 30911 tablet_bootstrap.cc:492] T b0e8b1909df34fce92f94d6af5f5c3b6 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:06.413224 30911 ts_tablet_manager.cc:1367] T b0e8b1909df34fce92f94d6af5f5c3b6 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:06.413727 30911 raft_consensus.cc:348] T b0e8b1909df34fce92f94d6af5f5c3b6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:06.413857 30911 raft_consensus.cc:374] T b0e8b1909df34fce92f94d6af5f5c3b6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:06.413900 30911 raft_consensus.cc:729] T b0e8b1909df34fce92f94d6af5f5c3b6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:06.413908 29162 tablet_service.cc:1467] Processing CreateTablet for tablet f65eb8cde25e483b9c790740ccfd357c (DEFAULT_TABLE table=rebalance_test_table_12 [id=e2320262ebe84e399b45c01bf9a7d73d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:06.414022 30911 consensus_queue.cc:260] T b0e8b1909df34fce92f94d6af5f5c3b6 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:06.414139 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f65eb8cde25e483b9c790740ccfd357c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:06.414343 30911 ts_tablet_manager.cc:1398] T b0e8b1909df34fce92f94d6af5f5c3b6 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 20:46:06.414577 29294 tablet_service.cc:1467] Processing CreateTablet for tablet f65eb8cde25e483b9c790740ccfd357c (DEFAULT_TABLE table=rebalance_test_table_12 [id=e2320262ebe84e399b45c01bf9a7d73d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:06.414597 29424 tablet_service.cc:1467] Processing CreateTablet for tablet 94bdf135dd3a47d39e7b76dbfd524386 (DEFAULT_TABLE table=rebalance_test_table_12 [id=e2320262ebe84e399b45c01bf9a7d73d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:06.414790 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f65eb8cde25e483b9c790740ccfd357c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:06.414834 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94bdf135dd3a47d39e7b76dbfd524386. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:06.415009 30913 tablet_bootstrap.cc:492] T f65eb8cde25e483b9c790740ccfd357c P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:06.415092 30913 ts_tablet_manager.cc:1367] T f65eb8cde25e483b9c790740ccfd357c P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 20:46:06.415556 30913 raft_consensus.cc:348] T f65eb8cde25e483b9c790740ccfd357c P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:06.415678 30913 raft_consensus.cc:374] T f65eb8cde25e483b9c790740ccfd357c P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:06.415736 30913 raft_consensus.cc:729] T f65eb8cde25e483b9c790740ccfd357c P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:06.415876 30913 consensus_queue.cc:260] T f65eb8cde25e483b9c790740ccfd357c P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:06.415936 30911 tablet_bootstrap.cc:492] T f65eb8cde25e483b9c790740ccfd357c P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:06.416172 30913 ts_tablet_manager.cc:1398] T f65eb8cde25e483b9c790740ccfd357c P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:06.416729 30911 tablet_bootstrap.cc:654] T f65eb8cde25e483b9c790740ccfd357c P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:06.416749 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 94bdf135dd3a47d39e7b76dbfd524386 (DEFAULT_TABLE table=rebalance_test_table_12 [id=e2320262ebe84e399b45c01bf9a7d73d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:06.416956 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94bdf135dd3a47d39e7b76dbfd524386. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:06.419454 30910 tablet_bootstrap.cc:492] T b0e8b1909df34fce92f94d6af5f5c3b6 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:06.419536 30910 ts_tablet_manager.cc:1367] T b0e8b1909df34fce92f94d6af5f5c3b6 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240502 20:46:06.420001 30910 raft_consensus.cc:348] T b0e8b1909df34fce92f94d6af5f5c3b6 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:06.420097 30910 raft_consensus.cc:374] T b0e8b1909df34fce92f94d6af5f5c3b6 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:06.420125 30910 raft_consensus.cc:729] T b0e8b1909df34fce92f94d6af5f5c3b6 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:06.420234 30910 consensus_queue.cc:260] T b0e8b1909df34fce92f94d6af5f5c3b6 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:06.420547 30910 ts_tablet_manager.cc:1398] T b0e8b1909df34fce92f94d6af5f5c3b6 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:06.420723 30910 tablet_bootstrap.cc:492] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:06.421469 30910 tablet_bootstrap.cc:654] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:06.422402 30910 tablet_bootstrap.cc:492] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:06.422520 30910 ts_tablet_manager.cc:1367] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:06.422904 30912 tablet_bootstrap.cc:492] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:06.422940 30910 raft_consensus.cc:348] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:06.422996 30912 ts_tablet_manager.cc:1367] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.002s
I20240502 20:46:06.423028 30910 raft_consensus.cc:374] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:06.423058 30910 raft_consensus.cc:729] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:06.423183 30910 consensus_queue.cc:260] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:06.423435 30910 ts_tablet_manager.cc:1398] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:06.423471 30912 raft_consensus.cc:348] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:06.423570 30910 tablet_bootstrap.cc:492] T 94bdf135dd3a47d39e7b76dbfd524386 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:06.423588 30912 raft_consensus.cc:374] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:06.423638 30912 raft_consensus.cc:729] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:06.423650 29557 tablet_service.cc:1467] Processing CreateTablet for tablet 94bdf135dd3a47d39e7b76dbfd524386 (DEFAULT_TABLE table=rebalance_test_table_12 [id=e2320262ebe84e399b45c01bf9a7d73d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:06.423766 30912 consensus_queue.cc:260] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:06.423869 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94bdf135dd3a47d39e7b76dbfd524386. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:06.424544 30910 tablet_bootstrap.cc:654] T 94bdf135dd3a47d39e7b76dbfd524386 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:06.424659 30912 ts_tablet_manager.cc:1398] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:06.424841 30912 tablet_bootstrap.cc:492] T 94bdf135dd3a47d39e7b76dbfd524386 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:06.425918 30912 tablet_bootstrap.cc:654] T 94bdf135dd3a47d39e7b76dbfd524386 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:06.427035 30912 tablet_bootstrap.cc:492] T 94bdf135dd3a47d39e7b76dbfd524386 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:06.427079 30911 tablet_bootstrap.cc:492] T f65eb8cde25e483b9c790740ccfd357c P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:06.427124 30912 ts_tablet_manager.cc:1367] T 94bdf135dd3a47d39e7b76dbfd524386 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:06.427165 30911 ts_tablet_manager.cc:1367] T f65eb8cde25e483b9c790740ccfd357c P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240502 20:46:06.427543 30912 raft_consensus.cc:348] T 94bdf135dd3a47d39e7b76dbfd524386 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:06.427632 30912 raft_consensus.cc:374] T 94bdf135dd3a47d39e7b76dbfd524386 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:06.427664 30912 raft_consensus.cc:729] T 94bdf135dd3a47d39e7b76dbfd524386 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:06.427639 30911 raft_consensus.cc:348] T f65eb8cde25e483b9c790740ccfd357c P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:06.427743 30911 raft_consensus.cc:374] T f65eb8cde25e483b9c790740ccfd357c P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:06.427779 30911 raft_consensus.cc:729] T f65eb8cde25e483b9c790740ccfd357c P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:06.427775 30912 consensus_queue.cc:260] T 94bdf135dd3a47d39e7b76dbfd524386 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:06.427855 30910 tablet_bootstrap.cc:492] T 94bdf135dd3a47d39e7b76dbfd524386 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:06.427927 30910 ts_tablet_manager.cc:1367] T 94bdf135dd3a47d39e7b76dbfd524386 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:06.427904 30911 consensus_queue.cc:260] T f65eb8cde25e483b9c790740ccfd357c P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:06.428042 30912 ts_tablet_manager.cc:1398] T 94bdf135dd3a47d39e7b76dbfd524386 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:06.428196 30911 ts_tablet_manager.cc:1398] T f65eb8cde25e483b9c790740ccfd357c P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:06.428385 30910 raft_consensus.cc:348] T 94bdf135dd3a47d39e7b76dbfd524386 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:06.428478 30910 raft_consensus.cc:374] T 94bdf135dd3a47d39e7b76dbfd524386 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:06.428516 30910 raft_consensus.cc:729] T 94bdf135dd3a47d39e7b76dbfd524386 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:06.428593 30913 tablet_bootstrap.cc:492] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:06.428639 30910 consensus_queue.cc:260] T 94bdf135dd3a47d39e7b76dbfd524386 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:06.428915 30910 ts_tablet_manager.cc:1398] T 94bdf135dd3a47d39e7b76dbfd524386 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:06.429476 30913 tablet_bootstrap.cc:654] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:06.430895 30913 tablet_bootstrap.cc:492] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:06.430989 30913 ts_tablet_manager.cc:1367] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:06.431358 30913 raft_consensus.cc:348] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:06.431463 30913 raft_consensus.cc:374] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:06.431507 30913 raft_consensus.cc:729] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:06.431650 30913 consensus_queue.cc:260] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:06.431936 30913 ts_tablet_manager.cc:1398] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:06.434212 30893 raft_consensus.cc:482] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:06.434294 30893 raft_consensus.cc:504] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:06.434727 30893 leader_election.cc:288] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:06.435074 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0e8b1909df34fce92f94d6af5f5c3b6" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:06.435101 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0e8b1909df34fce92f94d6af5f5c3b6" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:06.435178 29314 raft_consensus.cc:2454] T b0e8b1909df34fce92f94d6af5f5c3b6 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:06.435235 29182 raft_consensus.cc:2454] T b0e8b1909df34fce92f94d6af5f5c3b6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:06.435465 29381 leader_election.cc:302] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:06.435673 30893 raft_consensus.cc:2781] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:06.435720 30893 raft_consensus.cc:482] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:06.435745 30893 raft_consensus.cc:3037] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:06.436559 30893 raft_consensus.cc:504] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:06.436841 30893 leader_election.cc:288] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:06.437115 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0e8b1909df34fce92f94d6af5f5c3b6" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:06.437162 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0e8b1909df34fce92f94d6af5f5c3b6" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:06.437232 29314 raft_consensus.cc:3037] T b0e8b1909df34fce92f94d6af5f5c3b6 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:06.437268 29182 raft_consensus.cc:3037] T b0e8b1909df34fce92f94d6af5f5c3b6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:06.438153 29182 raft_consensus.cc:2454] T b0e8b1909df34fce92f94d6af5f5c3b6 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:06.438150 29314 raft_consensus.cc:2454] T b0e8b1909df34fce92f94d6af5f5c3b6 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:06.438423 29382 leader_election.cc:302] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:06.438602 30893 raft_consensus.cc:2781] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:06.438680 30893 raft_consensus.cc:686] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:06.438802 30893 consensus_queue.cc:237] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:06.440171 29034 catalog_manager.cc:5484] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:06.491083 30513 raft_consensus.cc:482] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:06.491179 30513 raft_consensus.cc:504] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:06.491484 30513 leader_election.cc:288] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:06.491796 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f65eb8cde25e483b9c790740ccfd357c" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:06.491817 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f65eb8cde25e483b9c790740ccfd357c" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:06.491943 29182 raft_consensus.cc:2454] T f65eb8cde25e483b9c790740ccfd357c P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:06.491945 29576 raft_consensus.cc:2454] T f65eb8cde25e483b9c790740ccfd357c P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:06.492213 29251 leader_election.cc:302] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:06.492417 30513 raft_consensus.cc:2781] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:06.492468 30513 raft_consensus.cc:482] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:06.492492 30513 raft_consensus.cc:3037] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:06.493216 30513 raft_consensus.cc:504] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:06.493506 30513 leader_election.cc:288] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:06.493814 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f65eb8cde25e483b9c790740ccfd357c" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:06.493809 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f65eb8cde25e483b9c790740ccfd357c" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:06.493904 29575 raft_consensus.cc:3037] T f65eb8cde25e483b9c790740ccfd357c P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:06.493916 29182 raft_consensus.cc:3037] T f65eb8cde25e483b9c790740ccfd357c P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:06.494603 29575 raft_consensus.cc:2454] T f65eb8cde25e483b9c790740ccfd357c P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:06.494738 29182 raft_consensus.cc:2454] T f65eb8cde25e483b9c790740ccfd357c P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:06.494874 29249 leader_election.cc:302] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:06.495013 30513 raft_consensus.cc:2781] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:06.495108 30513 raft_consensus.cc:686] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:06.495222 30513 consensus_queue.cc:237] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:06.496734 29034 catalog_manager.cc:5484] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:06.500463 30965 raft_consensus.cc:482] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:06.500548 30965 raft_consensus.cc:504] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:06.500902 30965 leader_election.cc:288] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:06.501209 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94bdf135dd3a47d39e7b76dbfd524386" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:06.501251 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94bdf135dd3a47d39e7b76dbfd524386" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:06.501338 29314 raft_consensus.cc:2454] T 94bdf135dd3a47d39e7b76dbfd524386 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:06.501334 29446 raft_consensus.cc:2454] T 94bdf135dd3a47d39e7b76dbfd524386 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:06.501572 29512 leader_election.cc:302] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:06.501732 30965 raft_consensus.cc:2781] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:06.501796 30965 raft_consensus.cc:482] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:06.501834 30965 raft_consensus.cc:3037] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:06.502643 30965 raft_consensus.cc:504] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:06.502945 30965 leader_election.cc:288] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Requested vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:06.503226 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94bdf135dd3a47d39e7b76dbfd524386" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:06.503304 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94bdf135dd3a47d39e7b76dbfd524386" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:06.503343 29446 raft_consensus.cc:3037] T 94bdf135dd3a47d39e7b76dbfd524386 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:06.503401 29314 raft_consensus.cc:3037] T 94bdf135dd3a47d39e7b76dbfd524386 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:06.504055 29314 raft_consensus.cc:2454] T 94bdf135dd3a47d39e7b76dbfd524386 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:06.504137 29446 raft_consensus.cc:2454] T 94bdf135dd3a47d39e7b76dbfd524386 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:06.504302 29512 leader_election.cc:302] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:06.504469 30965 raft_consensus.cc:2781] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:06.504555 30965 raft_consensus.cc:686] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Becoming Leader. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Running, Role: LEADER
I20240502 20:46:06.504673 30965 consensus_queue.cc:237] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:06.506255 29034 catalog_manager.cc:5484] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: term changed from 0 to 1, leader changed from <none> to db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
W20240502 20:46:06.625013 30978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:06.625228 30978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:06.628603 30978 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_12'
I20240502 20:46:06.705161 29314 raft_consensus.cc:1261] T b0e8b1909df34fce92f94d6af5f5c3b6 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:06.705545 29182 raft_consensus.cc:1261] T b0e8b1909df34fce92f94d6af5f5c3b6 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:06.705919 30893 consensus_queue.cc:1035] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:06.706063 29446 raft_consensus.cc:1261] T 94bdf135dd3a47d39e7b76dbfd524386 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:06.706521 30965 consensus_queue.cc:1035] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:06.706632 29182 raft_consensus.cc:1261] T f65eb8cde25e483b9c790740ccfd357c P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:06.707048 30591 consensus_queue.cc:1035] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:06.707202 30892 consensus_queue.cc:1035] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:06.711199 29313 raft_consensus.cc:1261] T 94bdf135dd3a47d39e7b76dbfd524386 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:06.712391 29575 raft_consensus.cc:1261] T f65eb8cde25e483b9c790740ccfd357c P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:06.713629 30965 consensus_queue.cc:1035] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:06.713969 30513 consensus_queue.cc:1035] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:06.720970 30881 mvcc.cc:205] Tried to move back new op lower bound from 7023340612420653056 to 7023340611334459392. Current Snapshot: MvccSnapshot[applied={T|T < 7023340612420653056}]
INSERT report
rows total: 2000
time total: 106.537 ms
time per row: 0.0532683 ms
I20240502 20:46:06.772262 29037 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalance_test_table_13"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:06.777313 29424 tablet_service.cc:1467] Processing CreateTablet for tablet 846f1d6e39e642cfaabf96ad0c52366d (DEFAULT_TABLE table=rebalance_test_table_13 [id=08ce6b9eef5e4b2da494180f1ee1852b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:06.777313 29162 tablet_service.cc:1467] Processing CreateTablet for tablet 846f1d6e39e642cfaabf96ad0c52366d (DEFAULT_TABLE table=rebalance_test_table_13 [id=08ce6b9eef5e4b2da494180f1ee1852b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:06.777593 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 846f1d6e39e642cfaabf96ad0c52366d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:06.777593 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 846f1d6e39e642cfaabf96ad0c52366d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:06.777912 29557 tablet_service.cc:1467] Processing CreateTablet for tablet 846f1d6e39e642cfaabf96ad0c52366d (DEFAULT_TABLE table=rebalance_test_table_13 [id=08ce6b9eef5e4b2da494180f1ee1852b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:06.778184 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 846f1d6e39e642cfaabf96ad0c52366d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:06.778388 29163 tablet_service.cc:1467] Processing CreateTablet for tablet 616e6b2cacc54145a29a10cc8584bbdf (DEFAULT_TABLE table=rebalance_test_table_13 [id=08ce6b9eef5e4b2da494180f1ee1852b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:06.778650 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 616e6b2cacc54145a29a10cc8584bbdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:06.779696 30911 tablet_bootstrap.cc:492] T 846f1d6e39e642cfaabf96ad0c52366d P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:06.780184 29424 tablet_service.cc:1467] Processing CreateTablet for tablet 616e6b2cacc54145a29a10cc8584bbdf (DEFAULT_TABLE table=rebalance_test_table_13 [id=08ce6b9eef5e4b2da494180f1ee1852b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:06.780345 29555 tablet_service.cc:1467] Processing CreateTablet for tablet 616e6b2cacc54145a29a10cc8584bbdf (DEFAULT_TABLE table=rebalance_test_table_13 [id=08ce6b9eef5e4b2da494180f1ee1852b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:06.780396 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 616e6b2cacc54145a29a10cc8584bbdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:06.780524 30911 tablet_bootstrap.cc:654] T 846f1d6e39e642cfaabf96ad0c52366d P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:06.780546 29555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 616e6b2cacc54145a29a10cc8584bbdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:06.780644 29164 tablet_service.cc:1467] Processing CreateTablet for tablet 21ee3d7c6f764d0483b335da40e61efc (DEFAULT_TABLE table=rebalance_test_table_13 [id=08ce6b9eef5e4b2da494180f1ee1852b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:06.780853 29164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21ee3d7c6f764d0483b335da40e61efc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:06.782351 30912 tablet_bootstrap.cc:492] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:06.783102 30912 tablet_bootstrap.cc:654] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:06.783754 29425 tablet_service.cc:1467] Processing CreateTablet for tablet 21ee3d7c6f764d0483b335da40e61efc (DEFAULT_TABLE table=rebalance_test_table_13 [id=08ce6b9eef5e4b2da494180f1ee1852b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:06.783939 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21ee3d7c6f764d0483b335da40e61efc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:06.784056 30912 tablet_bootstrap.cc:492] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:06.784118 30912 ts_tablet_manager.cc:1367] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:06.784437 30912 raft_consensus.cc:348] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:06.784509 30912 raft_consensus.cc:374] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:06.784528 30912 raft_consensus.cc:729] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:06.784622 30912 consensus_queue.cc:260] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:06.784883 30912 ts_tablet_manager.cc:1398] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:06.785094 30912 tablet_bootstrap.cc:492] T 616e6b2cacc54145a29a10cc8584bbdf P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:06.785490 30913 tablet_bootstrap.cc:492] T 846f1d6e39e642cfaabf96ad0c52366d P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:06.785857 30912 tablet_bootstrap.cc:654] T 616e6b2cacc54145a29a10cc8584bbdf P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:06.786331 30913 tablet_bootstrap.cc:654] T 846f1d6e39e642cfaabf96ad0c52366d P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:06.786808 30912 tablet_bootstrap.cc:492] T 616e6b2cacc54145a29a10cc8584bbdf P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:06.786875 30912 ts_tablet_manager.cc:1367] T 616e6b2cacc54145a29a10cc8584bbdf P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:06.787127 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 21ee3d7c6f764d0483b335da40e61efc (DEFAULT_TABLE table=rebalance_test_table_13 [id=08ce6b9eef5e4b2da494180f1ee1852b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:06.787223 30912 raft_consensus.cc:348] T 616e6b2cacc54145a29a10cc8584bbdf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:06.787295 30912 raft_consensus.cc:374] T 616e6b2cacc54145a29a10cc8584bbdf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:06.787313 30912 raft_consensus.cc:729] T 616e6b2cacc54145a29a10cc8584bbdf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:06.787357 30913 tablet_bootstrap.cc:492] T 846f1d6e39e642cfaabf96ad0c52366d P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:06.787379 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21ee3d7c6f764d0483b335da40e61efc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:06.787381 30912 consensus_queue.cc:260] T 616e6b2cacc54145a29a10cc8584bbdf P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:06.787425 30913 ts_tablet_manager.cc:1367] T 846f1d6e39e642cfaabf96ad0c52366d P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:06.787582 30912 ts_tablet_manager.cc:1398] T 616e6b2cacc54145a29a10cc8584bbdf P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:06.787704 30912 tablet_bootstrap.cc:492] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:06.787839 30913 raft_consensus.cc:348] T 846f1d6e39e642cfaabf96ad0c52366d P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:06.787923 30913 raft_consensus.cc:374] T 846f1d6e39e642cfaabf96ad0c52366d P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:06.787946 30913 raft_consensus.cc:729] T 846f1d6e39e642cfaabf96ad0c52366d P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:06.788056 30913 consensus_queue.cc:260] T 846f1d6e39e642cfaabf96ad0c52366d P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:06.788321 30913 ts_tablet_manager.cc:1398] T 846f1d6e39e642cfaabf96ad0c52366d P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:06.788370 30912 tablet_bootstrap.cc:654] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:06.788491 30913 tablet_bootstrap.cc:492] T 616e6b2cacc54145a29a10cc8584bbdf P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:06.789148 30910 tablet_bootstrap.cc:492] T 21ee3d7c6f764d0483b335da40e61efc P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:06.789330 30913 tablet_bootstrap.cc:654] T 616e6b2cacc54145a29a10cc8584bbdf P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:06.789920 30910 tablet_bootstrap.cc:654] T 21ee3d7c6f764d0483b335da40e61efc P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:06.790251 30913 tablet_bootstrap.cc:492] T 616e6b2cacc54145a29a10cc8584bbdf P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:06.790395 30913 ts_tablet_manager.cc:1367] T 616e6b2cacc54145a29a10cc8584bbdf P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:06.790899 30913 raft_consensus.cc:348] T 616e6b2cacc54145a29a10cc8584bbdf P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:06.791025 30910 tablet_bootstrap.cc:492] T 21ee3d7c6f764d0483b335da40e61efc P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:06.791056 30913 raft_consensus.cc:374] T 616e6b2cacc54145a29a10cc8584bbdf P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:06.791113 30910 ts_tablet_manager.cc:1367] T 21ee3d7c6f764d0483b335da40e61efc P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:06.791179 30913 raft_consensus.cc:729] T 616e6b2cacc54145a29a10cc8584bbdf P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:06.791334 30913 consensus_queue.cc:260] T 616e6b2cacc54145a29a10cc8584bbdf P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:06.791522 30910 raft_consensus.cc:348] T 21ee3d7c6f764d0483b335da40e61efc P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:06.791640 30910 raft_consensus.cc:374] T 21ee3d7c6f764d0483b335da40e61efc P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:06.791679 30910 raft_consensus.cc:729] T 21ee3d7c6f764d0483b335da40e61efc P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:06.791838 30913 ts_tablet_manager.cc:1398] T 616e6b2cacc54145a29a10cc8584bbdf P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:06.791805 30910 consensus_queue.cc:260] T 21ee3d7c6f764d0483b335da40e61efc P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:06.791980 30911 tablet_bootstrap.cc:492] T 846f1d6e39e642cfaabf96ad0c52366d P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:06.792066 30911 ts_tablet_manager.cc:1367] T 846f1d6e39e642cfaabf96ad0c52366d P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20240502 20:46:06.792079 30910 ts_tablet_manager.cc:1398] T 21ee3d7c6f764d0483b335da40e61efc P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:06.792519 30911 raft_consensus.cc:348] T 846f1d6e39e642cfaabf96ad0c52366d P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:06.792629 30911 raft_consensus.cc:374] T 846f1d6e39e642cfaabf96ad0c52366d P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:06.792659 30911 raft_consensus.cc:729] T 846f1d6e39e642cfaabf96ad0c52366d P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:06.792768 30911 consensus_queue.cc:260] T 846f1d6e39e642cfaabf96ad0c52366d P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:06.793040 30911 ts_tablet_manager.cc:1398] T 846f1d6e39e642cfaabf96ad0c52366d P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:06.793184 30911 tablet_bootstrap.cc:492] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:06.793869 30911 tablet_bootstrap.cc:654] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:06.794860 30911 tablet_bootstrap.cc:492] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:06.794919 30911 ts_tablet_manager.cc:1367] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:06.795212 30911 raft_consensus.cc:348] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:06.795318 30911 raft_consensus.cc:374] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:06.795361 30911 raft_consensus.cc:729] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:06.795527 30911 consensus_queue.cc:260] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:06.795845 30911 ts_tablet_manager.cc:1398] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:06.795914 30912 tablet_bootstrap.cc:492] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:06.795986 30912 ts_tablet_manager.cc:1367] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240502 20:46:06.795987 30911 tablet_bootstrap.cc:492] T 21ee3d7c6f764d0483b335da40e61efc P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:06.796387 30912 raft_consensus.cc:348] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:06.796491 30912 raft_consensus.cc:374] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:06.796525 30912 raft_consensus.cc:729] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:06.796628 30912 consensus_queue.cc:260] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:06.796785 30911 tablet_bootstrap.cc:654] T 21ee3d7c6f764d0483b335da40e61efc P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:06.796898 30912 ts_tablet_manager.cc:1398] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:06.798718 30911 tablet_bootstrap.cc:492] T 21ee3d7c6f764d0483b335da40e61efc P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:06.798804 30911 ts_tablet_manager.cc:1367] T 21ee3d7c6f764d0483b335da40e61efc P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:06.799192 30911 raft_consensus.cc:348] T 21ee3d7c6f764d0483b335da40e61efc P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:06.799280 30911 raft_consensus.cc:374] T 21ee3d7c6f764d0483b335da40e61efc P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:06.799309 30911 raft_consensus.cc:729] T 21ee3d7c6f764d0483b335da40e61efc P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:06.799429 30911 consensus_queue.cc:260] T 21ee3d7c6f764d0483b335da40e61efc P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:06.799753 30911 ts_tablet_manager.cc:1398] T 21ee3d7c6f764d0483b335da40e61efc P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:06.836002 31016 raft_consensus.cc:482] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:06.836124 31016 raft_consensus.cc:504] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:06.836520 31016 leader_election.cc:288] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:06.836865 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ee3d7c6f764d0483b335da40e61efc" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:06.836879 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ee3d7c6f764d0483b335da40e61efc" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:06.837002 29182 raft_consensus.cc:2454] T 21ee3d7c6f764d0483b335da40e61efc P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:06.837013 29314 raft_consensus.cc:2454] T 21ee3d7c6f764d0483b335da40e61efc P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:06.837266 29382 leader_election.cc:302] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:06.837446 31016 raft_consensus.cc:2781] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:06.837512 31016 raft_consensus.cc:482] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:06.837551 31016 raft_consensus.cc:3037] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:06.838613 31016 raft_consensus.cc:504] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:06.838963 31016 leader_election.cc:288] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:06.839270 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ee3d7c6f764d0483b335da40e61efc" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:06.839329 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ee3d7c6f764d0483b335da40e61efc" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:06.839378 29182 raft_consensus.cc:3037] T 21ee3d7c6f764d0483b335da40e61efc P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:06.839440 29314 raft_consensus.cc:3037] T 21ee3d7c6f764d0483b335da40e61efc P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:06.840158 29182 raft_consensus.cc:2454] T 21ee3d7c6f764d0483b335da40e61efc P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:06.840337 29314 raft_consensus.cc:2454] T 21ee3d7c6f764d0483b335da40e61efc P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:06.840621 29381 leader_election.cc:302] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:06.840787 31016 raft_consensus.cc:2781] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:06.840878 31016 raft_consensus.cc:686] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:06.841178 31016 consensus_queue.cc:237] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:06.843075 29033 catalog_manager.cc:5484] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:06.890998 31016 raft_consensus.cc:482] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:06.891121 31016 raft_consensus.cc:504] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:06.891481 31016 leader_election.cc:288] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:06.891850 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "846f1d6e39e642cfaabf96ad0c52366d" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:06.891898 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "846f1d6e39e642cfaabf96ad0c52366d" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:06.891985 29182 raft_consensus.cc:2454] T 846f1d6e39e642cfaabf96ad0c52366d P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:06.892067 29575 raft_consensus.cc:2454] T 846f1d6e39e642cfaabf96ad0c52366d P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:06.892306 29382 leader_election.cc:302] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:06.892489 31016 raft_consensus.cc:2781] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:06.892551 31016 raft_consensus.cc:482] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:06.892591 31016 raft_consensus.cc:3037] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:06.893349 31016 raft_consensus.cc:504] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:06.893658 31016 leader_election.cc:288] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:06.894006 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "846f1d6e39e642cfaabf96ad0c52366d" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:06.894006 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "846f1d6e39e642cfaabf96ad0c52366d" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:06.894132 29575 raft_consensus.cc:3037] T 846f1d6e39e642cfaabf96ad0c52366d P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:06.894132 29182 raft_consensus.cc:3037] T 846f1d6e39e642cfaabf96ad0c52366d P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:06.895076 29182 raft_consensus.cc:2454] T 846f1d6e39e642cfaabf96ad0c52366d P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:06.895076 29575 raft_consensus.cc:2454] T 846f1d6e39e642cfaabf96ad0c52366d P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:06.895383 29382 leader_election.cc:302] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:06.895570 31016 raft_consensus.cc:2781] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:06.895663 31016 raft_consensus.cc:686] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:06.895954 31016 consensus_queue.cc:237] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:06.897823 29033 catalog_manager.cc:5484] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:06.908129 30916 raft_consensus.cc:482] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:06.908226 30916 raft_consensus.cc:504] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:06.908553 30916 leader_election.cc:288] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:06.908910 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "616e6b2cacc54145a29a10cc8584bbdf" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:06.908910 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "616e6b2cacc54145a29a10cc8584bbdf" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:06.909047 29575 raft_consensus.cc:2454] T 616e6b2cacc54145a29a10cc8584bbdf P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:06.909047 29446 raft_consensus.cc:2454] T 616e6b2cacc54145a29a10cc8584bbdf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:06.909325 29118 leader_election.cc:302] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:06.909500 30916 raft_consensus.cc:2781] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:06.909559 30916 raft_consensus.cc:482] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:06.909584 30916 raft_consensus.cc:3037] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:06.910349 30916 raft_consensus.cc:504] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:06.910689 30916 leader_election.cc:288] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:06.911000 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "616e6b2cacc54145a29a10cc8584bbdf" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:06.911003 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "616e6b2cacc54145a29a10cc8584bbdf" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:06.911101 29575 raft_consensus.cc:3037] T 616e6b2cacc54145a29a10cc8584bbdf P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:06.911104 29446 raft_consensus.cc:3037] T 616e6b2cacc54145a29a10cc8584bbdf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:06.911871 29446 raft_consensus.cc:2454] T 616e6b2cacc54145a29a10cc8584bbdf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:06.911988 29575 raft_consensus.cc:2454] T 616e6b2cacc54145a29a10cc8584bbdf P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:06.912242 29119 leader_election.cc:302] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:06.912463 30916 raft_consensus.cc:2781] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:06.912550 30916 raft_consensus.cc:686] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:06.912679 30916 consensus_queue.cc:237] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:06.914415 29033 catalog_manager.cc:5484] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
W20240502 20:46:07.040313 31031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:07.040525 31031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:07.043591 31031 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_13'
I20240502 20:46:07.112661 29575 raft_consensus.cc:1261] T 846f1d6e39e642cfaabf96ad0c52366d P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:07.113091 30893 consensus_queue.cc:1035] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:07.113396 29314 raft_consensus.cc:1261] T 21ee3d7c6f764d0483b335da40e61efc P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:07.114638 29182 raft_consensus.cc:1261] T 21ee3d7c6f764d0483b335da40e61efc P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:07.115499 31016 consensus_queue.cc:1035] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:07.115779 31016 consensus_queue.cc:1035] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:07.119001 29576 raft_consensus.cc:1261] T 616e6b2cacc54145a29a10cc8584bbdf P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:07.119676 30916 consensus_queue.cc:1035] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:07.120433 29446 raft_consensus.cc:1261] T 616e6b2cacc54145a29a10cc8584bbdf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:07.120432 29175 raft_consensus.cc:1261] T 846f1d6e39e642cfaabf96ad0c52366d P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:07.121125 30756 consensus_queue.cc:1035] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:07.121587 30893 consensus_queue.cc:1035] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 94.2341 ms
time per row: 0.0471171 ms
I20240502 20:46:07.176759 29037 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalance_test_table_14"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:07.182547 29555 tablet_service.cc:1467] Processing CreateTablet for tablet e12d4c6713ca4606a1bc6ef41540dad6 (DEFAULT_TABLE table=rebalance_test_table_14 [id=b209cd0423394c4ea3701bdf9e25d73f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:07.182653 29294 tablet_service.cc:1467] Processing CreateTablet for tablet cd3fb6362a3445208fc3ceec878b9dcf (DEFAULT_TABLE table=rebalance_test_table_14 [id=b209cd0423394c4ea3701bdf9e25d73f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:07.182860 29555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e12d4c6713ca4606a1bc6ef41540dad6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:07.182893 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd3fb6362a3445208fc3ceec878b9dcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:07.183230 29164 tablet_service.cc:1467] Processing CreateTablet for tablet e12d4c6713ca4606a1bc6ef41540dad6 (DEFAULT_TABLE table=rebalance_test_table_14 [id=b209cd0423394c4ea3701bdf9e25d73f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:07.183481 29164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e12d4c6713ca4606a1bc6ef41540dad6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:07.185132 29295 tablet_service.cc:1467] Processing CreateTablet for tablet e12d4c6713ca4606a1bc6ef41540dad6 (DEFAULT_TABLE table=rebalance_test_table_14 [id=b209cd0423394c4ea3701bdf9e25d73f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:07.185349 30910 tablet_bootstrap.cc:492] T cd3fb6362a3445208fc3ceec878b9dcf P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:07.185367 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e12d4c6713ca4606a1bc6ef41540dad6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:07.185580 29163 tablet_service.cc:1467] Processing CreateTablet for tablet 37a946abe45e415892ac408cf8489d82 (DEFAULT_TABLE table=rebalance_test_table_14 [id=b209cd0423394c4ea3701bdf9e25d73f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:07.185804 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37a946abe45e415892ac408cf8489d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:07.186300 30910 tablet_bootstrap.cc:654] T cd3fb6362a3445208fc3ceec878b9dcf P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:07.186589 30913 tablet_bootstrap.cc:492] T e12d4c6713ca4606a1bc6ef41540dad6 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:07.187397 30913 tablet_bootstrap.cc:654] T e12d4c6713ca4606a1bc6ef41540dad6 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:07.187559 29557 tablet_service.cc:1467] Processing CreateTablet for tablet 37a946abe45e415892ac408cf8489d82 (DEFAULT_TABLE table=rebalance_test_table_14 [id=b209cd0423394c4ea3701bdf9e25d73f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:07.187608 30911 tablet_bootstrap.cc:492] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:07.187773 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37a946abe45e415892ac408cf8489d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:07.188373 30911 tablet_bootstrap.cc:654] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:07.188908 29555 tablet_service.cc:1467] Processing CreateTablet for tablet cd3fb6362a3445208fc3ceec878b9dcf (DEFAULT_TABLE table=rebalance_test_table_14 [id=b209cd0423394c4ea3701bdf9e25d73f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:07.189106 29555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd3fb6362a3445208fc3ceec878b9dcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:07.189628 29425 tablet_service.cc:1467] Processing CreateTablet for tablet cd3fb6362a3445208fc3ceec878b9dcf (DEFAULT_TABLE table=rebalance_test_table_14 [id=b209cd0423394c4ea3701bdf9e25d73f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:07.189862 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd3fb6362a3445208fc3ceec878b9dcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:07.191145 30913 tablet_bootstrap.cc:492] T e12d4c6713ca4606a1bc6ef41540dad6 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:07.191228 30913 ts_tablet_manager.cc:1367] T e12d4c6713ca4606a1bc6ef41540dad6 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20240502 20:46:07.191565 30913 raft_consensus.cc:348] T e12d4c6713ca4606a1bc6ef41540dad6 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:07.191651 30913 raft_consensus.cc:374] T e12d4c6713ca4606a1bc6ef41540dad6 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:07.191679 30913 raft_consensus.cc:729] T e12d4c6713ca4606a1bc6ef41540dad6 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:07.191800 30912 tablet_bootstrap.cc:492] T cd3fb6362a3445208fc3ceec878b9dcf P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:07.191784 30913 consensus_queue.cc:260] T e12d4c6713ca4606a1bc6ef41540dad6 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:07.192067 30913 ts_tablet_manager.cc:1398] T e12d4c6713ca4606a1bc6ef41540dad6 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:07.192231 30913 tablet_bootstrap.cc:492] T 37a946abe45e415892ac408cf8489d82 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:07.192574 30912 tablet_bootstrap.cc:654] T cd3fb6362a3445208fc3ceec878b9dcf P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:07.193068 30913 tablet_bootstrap.cc:654] T 37a946abe45e415892ac408cf8489d82 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:07.193549 30912 tablet_bootstrap.cc:492] T cd3fb6362a3445208fc3ceec878b9dcf P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:07.193624 30912 ts_tablet_manager.cc:1367] T cd3fb6362a3445208fc3ceec878b9dcf P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:07.194058 30912 raft_consensus.cc:348] T cd3fb6362a3445208fc3ceec878b9dcf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:07.194154 30912 raft_consensus.cc:374] T cd3fb6362a3445208fc3ceec878b9dcf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:07.194183 30912 raft_consensus.cc:729] T cd3fb6362a3445208fc3ceec878b9dcf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:07.194284 30912 consensus_queue.cc:260] T cd3fb6362a3445208fc3ceec878b9dcf P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:07.194340 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 37a946abe45e415892ac408cf8489d82 (DEFAULT_TABLE table=rebalance_test_table_14 [id=b209cd0423394c4ea3701bdf9e25d73f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:07.194563 30912 ts_tablet_manager.cc:1398] T cd3fb6362a3445208fc3ceec878b9dcf P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:07.194747 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37a946abe45e415892ac408cf8489d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:07.196473 30913 tablet_bootstrap.cc:492] T 37a946abe45e415892ac408cf8489d82 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:07.196550 30913 ts_tablet_manager.cc:1367] T 37a946abe45e415892ac408cf8489d82 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240502 20:46:07.196985 30913 raft_consensus.cc:348] T 37a946abe45e415892ac408cf8489d82 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:07.197127 30913 raft_consensus.cc:374] T 37a946abe45e415892ac408cf8489d82 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:07.197194 30913 raft_consensus.cc:729] T 37a946abe45e415892ac408cf8489d82 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:07.197361 30913 consensus_queue.cc:260] T 37a946abe45e415892ac408cf8489d82 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:07.197652 30913 ts_tablet_manager.cc:1398] T 37a946abe45e415892ac408cf8489d82 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:07.199533 30910 tablet_bootstrap.cc:492] T cd3fb6362a3445208fc3ceec878b9dcf P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:07.199532 30911 tablet_bootstrap.cc:492] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:07.199611 30911 ts_tablet_manager.cc:1367] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20240502 20:46:07.199615 30910 ts_tablet_manager.cc:1367] T cd3fb6362a3445208fc3ceec878b9dcf P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20240502 20:46:07.199995 30910 raft_consensus.cc:348] T cd3fb6362a3445208fc3ceec878b9dcf P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:07.200016 30911 raft_consensus.cc:348] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:07.200080 30910 raft_consensus.cc:374] T cd3fb6362a3445208fc3ceec878b9dcf P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:07.200105 30911 raft_consensus.cc:374] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:07.200110 30910 raft_consensus.cc:729] T cd3fb6362a3445208fc3ceec878b9dcf P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:07.200138 30911 raft_consensus.cc:729] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:07.200212 30910 consensus_queue.cc:260] T cd3fb6362a3445208fc3ceec878b9dcf P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:07.200235 30911 consensus_queue.cc:260] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:07.200460 30910 ts_tablet_manager.cc:1398] T cd3fb6362a3445208fc3ceec878b9dcf P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:07.200484 30911 ts_tablet_manager.cc:1398] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:07.200608 30910 tablet_bootstrap.cc:492] T e12d4c6713ca4606a1bc6ef41540dad6 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:07.200642 30911 tablet_bootstrap.cc:492] T 37a946abe45e415892ac408cf8489d82 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:07.201337 30910 tablet_bootstrap.cc:654] T e12d4c6713ca4606a1bc6ef41540dad6 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:07.201437 30911 tablet_bootstrap.cc:654] T 37a946abe45e415892ac408cf8489d82 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:07.202282 30910 tablet_bootstrap.cc:492] T e12d4c6713ca4606a1bc6ef41540dad6 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:07.202359 30910 ts_tablet_manager.cc:1367] T e12d4c6713ca4606a1bc6ef41540dad6 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:07.203292 30913 tablet_bootstrap.cc:492] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:07.204134 30913 tablet_bootstrap.cc:654] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:07.204862 30910 raft_consensus.cc:348] T e12d4c6713ca4606a1bc6ef41540dad6 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:07.204970 30910 raft_consensus.cc:374] T e12d4c6713ca4606a1bc6ef41540dad6 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:07.205010 30910 raft_consensus.cc:729] T e12d4c6713ca4606a1bc6ef41540dad6 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:07.205273 30910 consensus_queue.cc:260] T e12d4c6713ca4606a1bc6ef41540dad6 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:07.205626 30910 ts_tablet_manager.cc:1398] T e12d4c6713ca4606a1bc6ef41540dad6 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240502 20:46:07.205783 30910 tablet_bootstrap.cc:492] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:07.205788 30911 tablet_bootstrap.cc:492] T 37a946abe45e415892ac408cf8489d82 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:07.205853 30911 ts_tablet_manager.cc:1367] T 37a946abe45e415892ac408cf8489d82 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 20:46:07.206199 30911 raft_consensus.cc:348] T 37a946abe45e415892ac408cf8489d82 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:07.206282 30911 raft_consensus.cc:374] T 37a946abe45e415892ac408cf8489d82 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:07.206308 30911 raft_consensus.cc:729] T 37a946abe45e415892ac408cf8489d82 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:07.206403 30911 consensus_queue.cc:260] T 37a946abe45e415892ac408cf8489d82 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:07.206652 30911 ts_tablet_manager.cc:1398] T 37a946abe45e415892ac408cf8489d82 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:07.207145 30910 tablet_bootstrap.cc:654] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:07.207559 30913 tablet_bootstrap.cc:492] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:07.207629 30913 ts_tablet_manager.cc:1367] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240502 20:46:07.208000 30913 raft_consensus.cc:348] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:07.208081 30913 raft_consensus.cc:374] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:07.208106 30913 raft_consensus.cc:729] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:07.208132 30910 tablet_bootstrap.cc:492] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:07.208185 30910 ts_tablet_manager.cc:1367] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:07.208199 30913 consensus_queue.cc:260] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:07.208406 30913 ts_tablet_manager.cc:1398] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:07.208473 30910 raft_consensus.cc:348] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:07.208544 30910 raft_consensus.cc:374] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:07.208562 30910 raft_consensus.cc:729] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:07.208678 30910 consensus_queue.cc:260] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:07.208995 30910 ts_tablet_manager.cc:1398] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:07.222815 30771 raft_consensus.cc:482] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:07.222915 30771 raft_consensus.cc:504] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:07.223294 30771 leader_election.cc:288] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:07.223590 29313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd3fb6362a3445208fc3ceec878b9dcf" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:07.223647 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd3fb6362a3445208fc3ceec878b9dcf" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:07.223704 29313 raft_consensus.cc:2454] T cd3fb6362a3445208fc3ceec878b9dcf P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:07.223800 29446 raft_consensus.cc:2454] T cd3fb6362a3445208fc3ceec878b9dcf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:07.223985 29512 leader_election.cc:302] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:07.224145 30771 raft_consensus.cc:2781] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:07.224205 30771 raft_consensus.cc:482] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:07.224248 30771 raft_consensus.cc:3037] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:07.225108 30771 raft_consensus.cc:504] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:07.225386 30771 leader_election.cc:288] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:07.225713 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd3fb6362a3445208fc3ceec878b9dcf" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:07.225734 29313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd3fb6362a3445208fc3ceec878b9dcf" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:07.225834 29446 raft_consensus.cc:3037] T cd3fb6362a3445208fc3ceec878b9dcf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:07.225845 29313 raft_consensus.cc:3037] T cd3fb6362a3445208fc3ceec878b9dcf P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:07.226641 29446 raft_consensus.cc:2454] T cd3fb6362a3445208fc3ceec878b9dcf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:07.226788 29313 raft_consensus.cc:2454] T cd3fb6362a3445208fc3ceec878b9dcf P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:07.226888 29512 leader_election.cc:302] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:07.227033 30771 raft_consensus.cc:2781] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:07.227111 30771 raft_consensus.cc:686] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Becoming Leader. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Running, Role: LEADER
I20240502 20:46:07.227238 30771 consensus_queue.cc:237] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:07.228803 29037 catalog_manager.cc:5484] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: term changed from 0 to 1, leader changed from <none> to db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:07.390406 30756 raft_consensus.cc:482] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:07.390544 30756 raft_consensus.cc:504] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:07.390923 30756 leader_election.cc:288] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:07.391153 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e12d4c6713ca4606a1bc6ef41540dad6" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:07.391196 29313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e12d4c6713ca4606a1bc6ef41540dad6" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:07.391289 29575 raft_consensus.cc:2454] T e12d4c6713ca4606a1bc6ef41540dad6 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:07.391295 29313 raft_consensus.cc:2454] T e12d4c6713ca4606a1bc6ef41540dad6 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:07.391534 29118 leader_election.cc:302] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:07.391718 30756 raft_consensus.cc:2781] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:07.391775 30756 raft_consensus.cc:482] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:07.391813 30756 raft_consensus.cc:3037] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:07.392712 30756 raft_consensus.cc:504] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:07.393003 30756 leader_election.cc:288] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:07.393261 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e12d4c6713ca4606a1bc6ef41540dad6" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:07.393376 29575 raft_consensus.cc:3037] T e12d4c6713ca4606a1bc6ef41540dad6 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:07.393705 29313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e12d4c6713ca4606a1bc6ef41540dad6" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:07.393810 29313 raft_consensus.cc:3037] T e12d4c6713ca4606a1bc6ef41540dad6 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:07.394351 29575 raft_consensus.cc:2454] T e12d4c6713ca4606a1bc6ef41540dad6 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:07.394668 29118 leader_election.cc:302] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:07.394752 29313 raft_consensus.cc:2454] T e12d4c6713ca4606a1bc6ef41540dad6 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:07.394846 30756 raft_consensus.cc:2781] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:07.395143 30756 raft_consensus.cc:686] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:07.395282 30756 consensus_queue.cc:237] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:07.396853 29037 catalog_manager.cc:5484] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:07.479174 30513 raft_consensus.cc:482] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:07.479283 30513 raft_consensus.cc:504] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:07.479640 30513 leader_election.cc:288] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:07.479997 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37a946abe45e415892ac408cf8489d82" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:07.479984 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37a946abe45e415892ac408cf8489d82" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:07.480116 29575 raft_consensus.cc:2454] T 37a946abe45e415892ac408cf8489d82 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:07.480123 29175 raft_consensus.cc:2454] T 37a946abe45e415892ac408cf8489d82 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:07.480383 29249 leader_election.cc:302] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:07.480526 30513 raft_consensus.cc:2781] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:07.480580 30513 raft_consensus.cc:482] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:07.480610 30513 raft_consensus.cc:3037] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:07.481351 30513 raft_consensus.cc:504] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:07.481664 30513 leader_election.cc:288] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:07.481951 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37a946abe45e415892ac408cf8489d82" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:07.481994 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37a946abe45e415892ac408cf8489d82" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:07.482053 29175 raft_consensus.cc:3037] T 37a946abe45e415892ac408cf8489d82 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:07.482089 29575 raft_consensus.cc:3037] T 37a946abe45e415892ac408cf8489d82 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:07.482858 29175 raft_consensus.cc:2454] T 37a946abe45e415892ac408cf8489d82 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:07.482928 29575 raft_consensus.cc:2454] T 37a946abe45e415892ac408cf8489d82 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:07.483183 29251 leader_election.cc:302] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:07.483348 30513 raft_consensus.cc:2781] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:07.483433 30513 raft_consensus.cc:686] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:07.483565 30513 consensus_queue.cc:237] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:07.485495 29037 catalog_manager.cc:5484] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:07.638852 30771 consensus_queue.cc:1035] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:07.641983 30965 consensus_queue.cc:1035] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 20:46:07.664049 31080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:07.664319 31080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:07.667407 31080 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_14'
I20240502 20:46:07.741348 29313 raft_consensus.cc:1261] T e12d4c6713ca4606a1bc6ef41540dad6 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:07.741982 30916 consensus_queue.cc:1035] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:07.744167 29175 raft_consensus.cc:1261] T 37a946abe45e415892ac408cf8489d82 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:07.744587 30591 consensus_queue.cc:1035] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:07.752655 29575 raft_consensus.cc:1261] T e12d4c6713ca4606a1bc6ef41540dad6 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:07.753008 30916 consensus_queue.cc:1035] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:07.753118 29575 raft_consensus.cc:1261] T 37a946abe45e415892ac408cf8489d82 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:07.753863 30513 consensus_queue.cc:1035] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:07.774581 31114 mvcc.cc:205] Tried to move back new op lower bound from 7023340616660426752 to 7023340615613841408. Current Snapshot: MvccSnapshot[applied={T|T < 7023340616660426752}]
INSERT report
rows total: 2000
time total: 122.075 ms
time per row: 0.0610377 ms
I20240502 20:46:07.832441 29036 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalance_test_table_15"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:07.839043 29555 tablet_service.cc:1467] Processing CreateTablet for tablet e7642b18bcbb467187fc8c8b924b79df (DEFAULT_TABLE table=rebalance_test_table_15 [id=02149b4452244d3fbab1649adf739573]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:07.839268 29557 tablet_service.cc:1467] Processing CreateTablet for tablet fbd2bd8e142741ba864e2f99610b39d6 (DEFAULT_TABLE table=rebalance_test_table_15 [id=02149b4452244d3fbab1649adf739573]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:07.839347 29555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7642b18bcbb467187fc8c8b924b79df. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:07.839525 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbd2bd8e142741ba864e2f99610b39d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:07.839622 29163 tablet_service.cc:1467] Processing CreateTablet for tablet 035762cba9014632ad520f4777544641 (DEFAULT_TABLE table=rebalance_test_table_15 [id=02149b4452244d3fbab1649adf739573]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:07.839877 29425 tablet_service.cc:1467] Processing CreateTablet for tablet 035762cba9014632ad520f4777544641 (DEFAULT_TABLE table=rebalance_test_table_15 [id=02149b4452244d3fbab1649adf739573]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:07.839987 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 035762cba9014632ad520f4777544641. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:07.840103 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 035762cba9014632ad520f4777544641. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:07.841751 29424 tablet_service.cc:1467] Processing CreateTablet for tablet e7642b18bcbb467187fc8c8b924b79df (DEFAULT_TABLE table=rebalance_test_table_15 [id=02149b4452244d3fbab1649adf739573]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:07.841921 31136 tablet_bootstrap.cc:492] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:07.841981 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7642b18bcbb467187fc8c8b924b79df. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:07.842309 29164 tablet_service.cc:1467] Processing CreateTablet for tablet fbd2bd8e142741ba864e2f99610b39d6 (DEFAULT_TABLE table=rebalance_test_table_15 [id=02149b4452244d3fbab1649adf739573]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:07.842618 29164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbd2bd8e142741ba864e2f99610b39d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:07.842948 31136 tablet_bootstrap.cc:654] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:07.844069 31136 tablet_bootstrap.cc:492] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:07.844070 29426 tablet_service.cc:1467] Processing CreateTablet for tablet fbd2bd8e142741ba864e2f99610b39d6 (DEFAULT_TABLE table=rebalance_test_table_15 [id=02149b4452244d3fbab1649adf739573]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:07.844146 31136 ts_tablet_manager.cc:1367] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:07.844283 29426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbd2bd8e142741ba864e2f99610b39d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:07.844620 31136 raft_consensus.cc:348] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:07.844749 31136 raft_consensus.cc:374] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:07.844800 31136 raft_consensus.cc:729] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:07.844926 31136 consensus_queue.cc:260] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:07.845227 31136 ts_tablet_manager.cc:1398] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:07.845432 31136 tablet_bootstrap.cc:492] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:07.846240 31136 tablet_bootstrap.cc:654] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:07.846626 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 035762cba9014632ad520f4777544641 (DEFAULT_TABLE table=rebalance_test_table_15 [id=02149b4452244d3fbab1649adf739573]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:07.846904 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 035762cba9014632ad520f4777544641. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:07.847223 31136 tablet_bootstrap.cc:492] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:07.847297 31136 ts_tablet_manager.cc:1367] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:07.847641 31136 raft_consensus.cc:348] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:07.847718 31136 raft_consensus.cc:374] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:07.847743 31136 raft_consensus.cc:729] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:07.847823 31136 consensus_queue.cc:260] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:07.848112 31136 ts_tablet_manager.cc:1398] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:07.849045 31141 tablet_bootstrap.cc:492] T 035762cba9014632ad520f4777544641 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:07.849189 29294 tablet_service.cc:1467] Processing CreateTablet for tablet e7642b18bcbb467187fc8c8b924b79df (DEFAULT_TABLE table=rebalance_test_table_15 [id=02149b4452244d3fbab1649adf739573]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:07.849388 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7642b18bcbb467187fc8c8b924b79df. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:07.849989 31141 tablet_bootstrap.cc:654] T 035762cba9014632ad520f4777544641 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:07.851047 31141 tablet_bootstrap.cc:492] T 035762cba9014632ad520f4777544641 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:07.851115 31141 ts_tablet_manager.cc:1367] T 035762cba9014632ad520f4777544641 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 20:46:07.851531 31141 raft_consensus.cc:348] T 035762cba9014632ad520f4777544641 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:07.851627 31141 raft_consensus.cc:374] T 035762cba9014632ad520f4777544641 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:07.851657 31141 raft_consensus.cc:729] T 035762cba9014632ad520f4777544641 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:07.851783 31141 consensus_queue.cc:260] T 035762cba9014632ad520f4777544641 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:07.852149 31141 ts_tablet_manager.cc:1398] T 035762cba9014632ad520f4777544641 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:07.852404 31141 tablet_bootstrap.cc:492] T e7642b18bcbb467187fc8c8b924b79df P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:07.853219 31141 tablet_bootstrap.cc:654] T e7642b18bcbb467187fc8c8b924b79df P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:07.854173 31141 tablet_bootstrap.cc:492] T e7642b18bcbb467187fc8c8b924b79df P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:07.854275 31141 ts_tablet_manager.cc:1367] T e7642b18bcbb467187fc8c8b924b79df P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:07.854419 31137 tablet_bootstrap.cc:492] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:07.854643 31138 tablet_bootstrap.cc:492] T 035762cba9014632ad520f4777544641 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:07.854720 31141 raft_consensus.cc:348] T e7642b18bcbb467187fc8c8b924b79df P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:07.854815 31141 raft_consensus.cc:374] T e7642b18bcbb467187fc8c8b924b79df P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:07.854848 31141 raft_consensus.cc:729] T e7642b18bcbb467187fc8c8b924b79df P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:07.854979 31141 consensus_queue.cc:260] T e7642b18bcbb467187fc8c8b924b79df P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:07.855258 31141 ts_tablet_manager.cc:1398] T e7642b18bcbb467187fc8c8b924b79df P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:07.855881 31120 raft_consensus.cc:482] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:07.855962 31120 raft_consensus.cc:504] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:07.856288 31120 leader_election.cc:288] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:07.857137 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7642b18bcbb467187fc8c8b924b79df" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:07.857247 29314 raft_consensus.cc:2454] T e7642b18bcbb467187fc8c8b924b79df P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:07.857044 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7642b18bcbb467187fc8c8b924b79df" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:07.857517 29512 leader_election.cc:302] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057; no voters:
W20240502 20:46:07.857739 29512 leader_election.cc:341] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): Illegal state: must be running to vote when last-logged opid is not known
I20240502 20:46:07.858151 31120 raft_consensus.cc:2781] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:07.858222 31120 raft_consensus.cc:482] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:07.858269 31120 raft_consensus.cc:3037] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:07.859144 31120 raft_consensus.cc:504] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:07.859454 31120 leader_election.cc:288] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Requested vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:07.861397 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7642b18bcbb467187fc8c8b924b79df" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
W20240502 20:46:07.861737 29512 leader_election.cc:341] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): Illegal state: must be running to vote when last-logged opid is not known
I20240502 20:46:07.862301 31138 tablet_bootstrap.cc:654] T 035762cba9014632ad520f4777544641 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:07.863009 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7642b18bcbb467187fc8c8b924b79df" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:07.863109 29314 raft_consensus.cc:3037] T e7642b18bcbb467187fc8c8b924b79df P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:07.863916 29314 raft_consensus.cc:2454] T e7642b18bcbb467187fc8c8b924b79df P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:07.863972 31137 tablet_bootstrap.cc:654] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:07.864508 29512 leader_election.cc:302] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057; no voters: dbcdc6ac635b44b185c3eba63f66e9d6
I20240502 20:46:07.864858 31120 raft_consensus.cc:2781] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:07.864940 31120 raft_consensus.cc:686] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Becoming Leader. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Running, Role: LEADER
I20240502 20:46:07.865056 31120 consensus_queue.cc:237] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:07.866919 29033 catalog_manager.cc:5484] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: term changed from 0 to 1, leader changed from <none> to db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:07.867096 31137 tablet_bootstrap.cc:492] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:07.868952 31138 tablet_bootstrap.cc:492] T 035762cba9014632ad520f4777544641 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:07.869055 31137 ts_tablet_manager.cc:1367] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.002s
I20240502 20:46:07.869657 31137 raft_consensus.cc:348] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:07.869833 31138 ts_tablet_manager.cc:1367] T 035762cba9014632ad520f4777544641 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 20:46:07.871183 31138 raft_consensus.cc:348] T 035762cba9014632ad520f4777544641 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:07.871342 31137 raft_consensus.cc:374] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:07.871430 31138 raft_consensus.cc:374] T 035762cba9014632ad520f4777544641 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:07.871510 31137 raft_consensus.cc:729] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:07.871692 31137 consensus_queue.cc:260] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:07.871837 31138 raft_consensus.cc:729] T 035762cba9014632ad520f4777544641 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:07.872021 31138 consensus_queue.cc:260] T 035762cba9014632ad520f4777544641 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:07.872373 31137 ts_tablet_manager.cc:1398] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240502 20:46:07.873608 31138 ts_tablet_manager.cc:1398] T 035762cba9014632ad520f4777544641 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:07.873843 31137 tablet_bootstrap.cc:492] T fbd2bd8e142741ba864e2f99610b39d6 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:07.874053 31138 tablet_bootstrap.cc:492] T e7642b18bcbb467187fc8c8b924b79df P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:07.874969 31137 tablet_bootstrap.cc:654] T fbd2bd8e142741ba864e2f99610b39d6 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:07.875758 31138 tablet_bootstrap.cc:654] T e7642b18bcbb467187fc8c8b924b79df P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:07.877589 31138 tablet_bootstrap.cc:492] T e7642b18bcbb467187fc8c8b924b79df P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:07.877701 31137 tablet_bootstrap.cc:492] T fbd2bd8e142741ba864e2f99610b39d6 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:07.877732 31138 ts_tablet_manager.cc:1367] T e7642b18bcbb467187fc8c8b924b79df P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:07.877790 31137 ts_tablet_manager.cc:1367] T fbd2bd8e142741ba864e2f99610b39d6 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:07.878141 31137 raft_consensus.cc:348] T fbd2bd8e142741ba864e2f99610b39d6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:07.878226 31137 raft_consensus.cc:374] T fbd2bd8e142741ba864e2f99610b39d6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:07.878250 31137 raft_consensus.cc:729] T fbd2bd8e142741ba864e2f99610b39d6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:07.878360 31137 consensus_queue.cc:260] T fbd2bd8e142741ba864e2f99610b39d6 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:07.878608 31137 ts_tablet_manager.cc:1398] T fbd2bd8e142741ba864e2f99610b39d6 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:07.881424 31138 raft_consensus.cc:348] T e7642b18bcbb467187fc8c8b924b79df P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:07.881565 31138 raft_consensus.cc:374] T e7642b18bcbb467187fc8c8b924b79df P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:07.881613 31138 raft_consensus.cc:729] T e7642b18bcbb467187fc8c8b924b79df P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:07.881745 31138 consensus_queue.cc:260] T e7642b18bcbb467187fc8c8b924b79df P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:07.882064 31138 ts_tablet_manager.cc:1398] T e7642b18bcbb467187fc8c8b924b79df P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:07.882246 31138 tablet_bootstrap.cc:492] T fbd2bd8e142741ba864e2f99610b39d6 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:07.884498 31138 tablet_bootstrap.cc:654] T fbd2bd8e142741ba864e2f99610b39d6 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:07.886145 31138 tablet_bootstrap.cc:492] T fbd2bd8e142741ba864e2f99610b39d6 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:07.886242 31138 ts_tablet_manager.cc:1367] T fbd2bd8e142741ba864e2f99610b39d6 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:07.886610 31138 raft_consensus.cc:348] T fbd2bd8e142741ba864e2f99610b39d6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:07.886708 31138 raft_consensus.cc:374] T fbd2bd8e142741ba864e2f99610b39d6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:07.886744 31138 raft_consensus.cc:729] T fbd2bd8e142741ba864e2f99610b39d6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:07.886864 31138 consensus_queue.cc:260] T fbd2bd8e142741ba864e2f99610b39d6 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:07.887115 31138 ts_tablet_manager.cc:1398] T fbd2bd8e142741ba864e2f99610b39d6 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:07.889954 30892 raft_consensus.cc:482] T e7642b18bcbb467187fc8c8b924b79df P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:07.890043 30892 raft_consensus.cc:504] T e7642b18bcbb467187fc8c8b924b79df P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:07.890378 30892 leader_election.cc:288] T e7642b18bcbb467187fc8c8b924b79df P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:07.890659 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7642b18bcbb467187fc8c8b924b79df" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:07.890683 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7642b18bcbb467187fc8c8b924b79df" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:07.890808 29314 raft_consensus.cc:2379] T e7642b18bcbb467187fc8c8b924b79df P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbcdc6ac635b44b185c3eba63f66e9d6 in current term 1: Already voted for candidate db17f26a73f3447b92aa9a947e3a7057 in this term.
I20240502 20:46:07.891290 29380 leader_election.cc:302] T e7642b18bcbb467187fc8c8b924b79df P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbcdc6ac635b44b185c3eba63f66e9d6; no voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057
I20240502 20:46:07.891453 30892 raft_consensus.cc:3037] T e7642b18bcbb467187fc8c8b924b79df P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:07.892252 30892 raft_consensus.cc:2726] T e7642b18bcbb467187fc8c8b924b79df P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 20:46:07.925448 30965 raft_consensus.cc:482] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:07.925536 30965 raft_consensus.cc:504] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:07.925846 30965 leader_election.cc:288] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:07.926023 30916 raft_consensus.cc:482] T fbd2bd8e142741ba864e2f99610b39d6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:07.926115 30916 raft_consensus.cc:504] T fbd2bd8e142741ba864e2f99610b39d6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:07.926123 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbd2bd8e142741ba864e2f99610b39d6" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:07.926213 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbd2bd8e142741ba864e2f99610b39d6" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:07.926239 29446 raft_consensus.cc:2454] T fbd2bd8e142741ba864e2f99610b39d6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:07.926296 29175 raft_consensus.cc:2454] T fbd2bd8e142741ba864e2f99610b39d6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:07.926600 29513 leader_election.cc:302] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:07.926784 30965 raft_consensus.cc:2781] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:07.926843 30965 raft_consensus.cc:482] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:07.926847 30916 leader_election.cc:288] T fbd2bd8e142741ba864e2f99610b39d6 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:07.926877 30965 raft_consensus.cc:3037] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:07.927002 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbd2bd8e142741ba864e2f99610b39d6" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:07.927223 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbd2bd8e142741ba864e2f99610b39d6" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:07.927341 29446 raft_consensus.cc:2454] T fbd2bd8e142741ba864e2f99610b39d6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:07.927577 29119 leader_election.cc:302] T fbd2bd8e142741ba864e2f99610b39d6 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:07.927740 30916 raft_consensus.cc:2781] T fbd2bd8e142741ba864e2f99610b39d6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:07.927793 30916 raft_consensus.cc:482] T fbd2bd8e142741ba864e2f99610b39d6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:07.927819 30916 raft_consensus.cc:3037] T fbd2bd8e142741ba864e2f99610b39d6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:07.927841 30965 raft_consensus.cc:504] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:07.928030 29575 raft_consensus.cc:2379] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7796ea3fba294d17a34386873969431d in current term 1: Already voted for candidate db17f26a73f3447b92aa9a947e3a7057 in this term.
I20240502 20:46:07.928179 30965 leader_election.cc:288] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Requested vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:07.928575 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbd2bd8e142741ba864e2f99610b39d6" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:07.928671 29446 raft_consensus.cc:3037] T fbd2bd8e142741ba864e2f99610b39d6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:07.928776 30916 raft_consensus.cc:504] T fbd2bd8e142741ba864e2f99610b39d6 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:07.929066 30916 leader_election.cc:288] T fbd2bd8e142741ba864e2f99610b39d6 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:07.929448 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbd2bd8e142741ba864e2f99610b39d6" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:07.929544 29446 raft_consensus.cc:2454] T fbd2bd8e142741ba864e2f99610b39d6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:07.929535 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbd2bd8e142741ba864e2f99610b39d6" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:07.929591 29175 raft_consensus.cc:2379] T fbd2bd8e142741ba864e2f99610b39d6 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate db17f26a73f3447b92aa9a947e3a7057 in current term 1: Already voted for candidate 7796ea3fba294d17a34386873969431d in this term.
I20240502 20:46:07.929665 29575 raft_consensus.cc:2379] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7796ea3fba294d17a34386873969431d in current term 1: Already voted for candidate db17f26a73f3447b92aa9a947e3a7057 in this term.
I20240502 20:46:07.929719 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbd2bd8e142741ba864e2f99610b39d6" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:07.929841 29444 raft_consensus.cc:2379] T fbd2bd8e142741ba864e2f99610b39d6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7796ea3fba294d17a34386873969431d in current term 1: Already voted for candidate db17f26a73f3447b92aa9a947e3a7057 in this term.
I20240502 20:46:07.930001 29512 leader_election.cc:302] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:07.930101 29119 leader_election.cc:302] T fbd2bd8e142741ba864e2f99610b39d6 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7796ea3fba294d17a34386873969431d; no voters: db17f26a73f3447b92aa9a947e3a7057, dbcdc6ac635b44b185c3eba63f66e9d6
I20240502 20:46:07.930405 31120 raft_consensus.cc:2781] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:07.930696 31120 raft_consensus.cc:686] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Becoming Leader. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Running, Role: LEADER
I20240502 20:46:07.930824 31120 consensus_queue.cc:237] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:07.932595 30916 raft_consensus.cc:2726] T fbd2bd8e142741ba864e2f99610b39d6 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240502 20:46:07.932646 29033 catalog_manager.cc:5484] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: term changed from 0 to 1, leader changed from <none> to db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:07.940160 30916 raft_consensus.cc:482] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:07.940238 30916 raft_consensus.cc:504] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:07.940546 30916 leader_election.cc:288] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:07.940774 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "035762cba9014632ad520f4777544641" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:07.940843 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "035762cba9014632ad520f4777544641" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:07.940901 29314 raft_consensus.cc:2454] T 035762cba9014632ad520f4777544641 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:07.940937 29446 raft_consensus.cc:2454] T 035762cba9014632ad520f4777544641 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:07.941133 29119 leader_election.cc:302] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:07.941279 30916 raft_consensus.cc:2781] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:07.941335 30916 raft_consensus.cc:482] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:07.941380 30916 raft_consensus.cc:3037] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:07.941964 30916 raft_consensus.cc:504] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:07.942224 30916 leader_election.cc:288] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:07.942535 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "035762cba9014632ad520f4777544641" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:07.942561 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "035762cba9014632ad520f4777544641" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:07.942646 29446 raft_consensus.cc:3037] T 035762cba9014632ad520f4777544641 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:07.942646 29314 raft_consensus.cc:3037] T 035762cba9014632ad520f4777544641 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:07.943432 29446 raft_consensus.cc:2454] T 035762cba9014632ad520f4777544641 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:07.943536 29314 raft_consensus.cc:2454] T 035762cba9014632ad520f4777544641 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:07.943724 29119 leader_election.cc:302] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:07.944069 30916 raft_consensus.cc:2781] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:07.944159 30916 raft_consensus.cc:686] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:07.944406 30916 consensus_queue.cc:237] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:07.946025 29033 catalog_manager.cc:5484] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
W20240502 20:46:08.057152 31152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:08.057428 31152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:08.061313 31152 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_15'
I20240502 20:46:08.149294 29314 raft_consensus.cc:1261] T e7642b18bcbb467187fc8c8b924b79df P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:08.150010 29175 raft_consensus.cc:1261] T fbd2bd8e142741ba864e2f99610b39d6 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:08.150239 29446 raft_consensus.cc:1261] T e7642b18bcbb467187fc8c8b924b79df P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:08.150283 29444 raft_consensus.cc:1261] T fbd2bd8e142741ba864e2f99610b39d6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:08.150552 30965 consensus_queue.cc:1035] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:08.151975 30965 consensus_queue.cc:1035] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:08.152067 31120 consensus_queue.cc:1035] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:08.152246 30965 consensus_queue.cc:1035] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:08.157133 29313 raft_consensus.cc:1261] T 035762cba9014632ad520f4777544641 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:08.157574 29445 raft_consensus.cc:1261] T 035762cba9014632ad520f4777544641 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:08.158208 30756 consensus_queue.cc:1035] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:08.158497 30756 consensus_queue.cc:1035] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 118.511 ms
time per row: 0.0592554 ms
I20240502 20:46:08.219105 29040 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalance_test_table_16"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:08.224380 29164 tablet_service.cc:1467] Processing CreateTablet for tablet e01c5b97e2d14976812691b78a5143c5 (DEFAULT_TABLE table=rebalance_test_table_16 [id=c5c351da1d6f466a88f1fcf68cd0a72d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:08.224658 29164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e01c5b97e2d14976812691b78a5143c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:08.225121 29294 tablet_service.cc:1467] Processing CreateTablet for tablet e01c5b97e2d14976812691b78a5143c5 (DEFAULT_TABLE table=rebalance_test_table_16 [id=c5c351da1d6f466a88f1fcf68cd0a72d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:08.225242 29293 tablet_service.cc:1467] Processing CreateTablet for tablet 791a0a29d67447fcb54e38e848680330 (DEFAULT_TABLE table=rebalance_test_table_16 [id=c5c351da1d6f466a88f1fcf68cd0a72d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:08.225386 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e01c5b97e2d14976812691b78a5143c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:08.225512 29293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 791a0a29d67447fcb54e38e848680330. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:08.225807 29426 tablet_service.cc:1467] Processing CreateTablet for tablet 1d0d00056f794ed89631137c89569385 (DEFAULT_TABLE table=rebalance_test_table_16 [id=c5c351da1d6f466a88f1fcf68cd0a72d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:08.226027 29426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d0d00056f794ed89631137c89569385. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:08.226985 29163 tablet_service.cc:1467] Processing CreateTablet for tablet 1d0d00056f794ed89631137c89569385 (DEFAULT_TABLE table=rebalance_test_table_16 [id=c5c351da1d6f466a88f1fcf68cd0a72d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:08.227200 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d0d00056f794ed89631137c89569385. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:08.227663 31141 tablet_bootstrap.cc:492] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:08.227998 31138 tablet_bootstrap.cc:492] T 1d0d00056f794ed89631137c89569385 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:08.228176 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 1d0d00056f794ed89631137c89569385 (DEFAULT_TABLE table=rebalance_test_table_16 [id=c5c351da1d6f466a88f1fcf68cd0a72d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:08.228384 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d0d00056f794ed89631137c89569385. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:08.228466 31141 tablet_bootstrap.cc:654] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:08.228817 31138 tablet_bootstrap.cc:654] T 1d0d00056f794ed89631137c89569385 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:08.228938 31137 tablet_bootstrap.cc:492] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:08.229660 31137 tablet_bootstrap.cc:654] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:08.229794 31138 tablet_bootstrap.cc:492] T 1d0d00056f794ed89631137c89569385 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:08.229871 31138 ts_tablet_manager.cc:1367] T 1d0d00056f794ed89631137c89569385 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:08.230266 31138 raft_consensus.cc:348] T 1d0d00056f794ed89631137c89569385 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:08.230381 31138 raft_consensus.cc:374] T 1d0d00056f794ed89631137c89569385 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:08.230348 29424 tablet_service.cc:1467] Processing CreateTablet for tablet 791a0a29d67447fcb54e38e848680330 (DEFAULT_TABLE table=rebalance_test_table_16 [id=c5c351da1d6f466a88f1fcf68cd0a72d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:08.230412 31138 raft_consensus.cc:729] T 1d0d00056f794ed89631137c89569385 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:08.230553 31138 consensus_queue.cc:260] T 1d0d00056f794ed89631137c89569385 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:08.230628 31137 tablet_bootstrap.cc:492] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:08.230691 31137 ts_tablet_manager.cc:1367] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:08.230840 31138 ts_tablet_manager.cc:1398] T 1d0d00056f794ed89631137c89569385 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:08.231050 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 791a0a29d67447fcb54e38e848680330. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:08.231246 31137 raft_consensus.cc:348] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:08.231330 31137 raft_consensus.cc:374] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:08.231350 31137 raft_consensus.cc:729] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:08.231444 31137 consensus_queue.cc:260] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:08.231678 31137 ts_tablet_manager.cc:1398] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:08.231833 31137 tablet_bootstrap.cc:492] T 1d0d00056f794ed89631137c89569385 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:08.232461 31137 tablet_bootstrap.cc:654] T 1d0d00056f794ed89631137c89569385 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:08.233556 31137 tablet_bootstrap.cc:492] T 1d0d00056f794ed89631137c89569385 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:08.233629 31137 ts_tablet_manager.cc:1367] T 1d0d00056f794ed89631137c89569385 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:08.233582 29557 tablet_service.cc:1467] Processing CreateTablet for tablet e01c5b97e2d14976812691b78a5143c5 (DEFAULT_TABLE table=rebalance_test_table_16 [id=c5c351da1d6f466a88f1fcf68cd0a72d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:08.233829 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e01c5b97e2d14976812691b78a5143c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:08.234011 31137 raft_consensus.cc:348] T 1d0d00056f794ed89631137c89569385 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:08.234104 31137 raft_consensus.cc:374] T 1d0d00056f794ed89631137c89569385 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:08.234135 31137 raft_consensus.cc:729] T 1d0d00056f794ed89631137c89569385 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:08.234230 31137 consensus_queue.cc:260] T 1d0d00056f794ed89631137c89569385 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:08.234534 31137 ts_tablet_manager.cc:1398] T 1d0d00056f794ed89631137c89569385 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:08.234696 29555 tablet_service.cc:1467] Processing CreateTablet for tablet 791a0a29d67447fcb54e38e848680330 (DEFAULT_TABLE table=rebalance_test_table_16 [id=c5c351da1d6f466a88f1fcf68cd0a72d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:08.234870 29555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 791a0a29d67447fcb54e38e848680330. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:08.235818 31136 tablet_bootstrap.cc:492] T e01c5b97e2d14976812691b78a5143c5 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:08.236570 31136 tablet_bootstrap.cc:654] T e01c5b97e2d14976812691b78a5143c5 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:08.237695 31138 tablet_bootstrap.cc:492] T 791a0a29d67447fcb54e38e848680330 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:08.237728 31136 tablet_bootstrap.cc:492] T e01c5b97e2d14976812691b78a5143c5 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:08.237802 31136 ts_tablet_manager.cc:1367] T e01c5b97e2d14976812691b78a5143c5 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:08.238202 31136 raft_consensus.cc:348] T e01c5b97e2d14976812691b78a5143c5 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:08.238291 31136 raft_consensus.cc:374] T e01c5b97e2d14976812691b78a5143c5 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:08.238320 31136 raft_consensus.cc:729] T e01c5b97e2d14976812691b78a5143c5 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:08.238512 31138 tablet_bootstrap.cc:654] T 791a0a29d67447fcb54e38e848680330 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:08.238433 31136 consensus_queue.cc:260] T e01c5b97e2d14976812691b78a5143c5 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:08.238772 31136 ts_tablet_manager.cc:1398] T e01c5b97e2d14976812691b78a5143c5 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:08.239351 31136 tablet_bootstrap.cc:492] T 791a0a29d67447fcb54e38e848680330 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:08.239503 31138 tablet_bootstrap.cc:492] T 791a0a29d67447fcb54e38e848680330 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:08.239573 31138 ts_tablet_manager.cc:1367] T 791a0a29d67447fcb54e38e848680330 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:08.239954 31138 raft_consensus.cc:348] T 791a0a29d67447fcb54e38e848680330 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:08.240039 31138 raft_consensus.cc:374] T 791a0a29d67447fcb54e38e848680330 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:08.240067 31138 raft_consensus.cc:729] T 791a0a29d67447fcb54e38e848680330 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:08.240118 31136 tablet_bootstrap.cc:654] T 791a0a29d67447fcb54e38e848680330 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:08.240165 31138 consensus_queue.cc:260] T 791a0a29d67447fcb54e38e848680330 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:08.240439 31138 ts_tablet_manager.cc:1398] T 791a0a29d67447fcb54e38e848680330 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:08.241240 31136 tablet_bootstrap.cc:492] T 791a0a29d67447fcb54e38e848680330 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:08.241420 31136 ts_tablet_manager.cc:1367] T 791a0a29d67447fcb54e38e848680330 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:08.241849 31136 raft_consensus.cc:348] T 791a0a29d67447fcb54e38e848680330 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:08.241997 31136 raft_consensus.cc:374] T 791a0a29d67447fcb54e38e848680330 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:08.242075 31136 raft_consensus.cc:729] T 791a0a29d67447fcb54e38e848680330 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:08.242216 31136 consensus_queue.cc:260] T 791a0a29d67447fcb54e38e848680330 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:08.242544 31136 ts_tablet_manager.cc:1398] T 791a0a29d67447fcb54e38e848680330 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:08.243592 31141 tablet_bootstrap.cc:492] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:08.243682 31141 ts_tablet_manager.cc:1367] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20240502 20:46:08.244109 31141 raft_consensus.cc:348] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:08.244215 31141 raft_consensus.cc:374] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:08.244264 31141 raft_consensus.cc:729] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:08.244376 31141 consensus_queue.cc:260] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:08.244638 31141 ts_tablet_manager.cc:1398] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:08.244792 31141 tablet_bootstrap.cc:492] T e01c5b97e2d14976812691b78a5143c5 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:08.245591 31141 tablet_bootstrap.cc:654] T e01c5b97e2d14976812691b78a5143c5 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:08.246810 31141 tablet_bootstrap.cc:492] T e01c5b97e2d14976812691b78a5143c5 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:08.246898 31141 ts_tablet_manager.cc:1367] T e01c5b97e2d14976812691b78a5143c5 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 20:46:08.248018 31141 raft_consensus.cc:348] T e01c5b97e2d14976812691b78a5143c5 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:08.248099 31141 raft_consensus.cc:374] T e01c5b97e2d14976812691b78a5143c5 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:08.248126 31141 raft_consensus.cc:729] T e01c5b97e2d14976812691b78a5143c5 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:08.248229 31141 consensus_queue.cc:260] T e01c5b97e2d14976812691b78a5143c5 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:08.248435 31141 ts_tablet_manager.cc:1398] T e01c5b97e2d14976812691b78a5143c5 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:08.248616 31141 tablet_bootstrap.cc:492] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:08.249408 31141 tablet_bootstrap.cc:654] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:08.250401 31141 tablet_bootstrap.cc:492] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:08.250550 31141 ts_tablet_manager.cc:1367] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:08.251150 31141 raft_consensus.cc:348] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:08.251250 31141 raft_consensus.cc:374] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:08.251291 31141 raft_consensus.cc:729] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:08.251391 31141 consensus_queue.cc:260] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:08.251644 31141 ts_tablet_manager.cc:1398] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:08.268324 30591 raft_consensus.cc:482] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:08.268395 30591 raft_consensus.cc:504] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:08.268694 30591 leader_election.cc:288] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:08.268972 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d0d00056f794ed89631137c89569385" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:08.268990 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d0d00056f794ed89631137c89569385" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:08.269069 29446 raft_consensus.cc:2454] T 1d0d00056f794ed89631137c89569385 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:08.269095 29175 raft_consensus.cc:2454] T 1d0d00056f794ed89631137c89569385 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:08.269330 29250 leader_election.cc:302] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:08.269515 30591 raft_consensus.cc:2781] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:08.269572 30591 raft_consensus.cc:482] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:08.269596 30591 raft_consensus.cc:3037] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:08.270213 30591 raft_consensus.cc:504] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:08.270526 30591 leader_election.cc:288] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:08.270802 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d0d00056f794ed89631137c89569385" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:08.270802 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d0d00056f794ed89631137c89569385" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:08.270910 29175 raft_consensus.cc:3037] T 1d0d00056f794ed89631137c89569385 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:08.270910 29446 raft_consensus.cc:3037] T 1d0d00056f794ed89631137c89569385 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:08.271791 29175 raft_consensus.cc:2454] T 1d0d00056f794ed89631137c89569385 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:08.271823 29446 raft_consensus.cc:2454] T 1d0d00056f794ed89631137c89569385 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:08.272110 29251 leader_election.cc:302] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:08.272266 30591 raft_consensus.cc:2781] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:08.272346 30591 raft_consensus.cc:686] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:08.272469 30591 consensus_queue.cc:237] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:08.273975 29037 catalog_manager.cc:5484] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:08.355458 30916 raft_consensus.cc:482] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:08.355578 30916 raft_consensus.cc:504] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:08.355938 30916 leader_election.cc:288] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:08.356252 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e01c5b97e2d14976812691b78a5143c5" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:08.356259 29313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e01c5b97e2d14976812691b78a5143c5" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:08.356395 29313 raft_consensus.cc:2454] T e01c5b97e2d14976812691b78a5143c5 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:08.356395 29577 raft_consensus.cc:2454] T e01c5b97e2d14976812691b78a5143c5 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:08.356653 29118 leader_election.cc:302] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:08.356854 30916 raft_consensus.cc:2781] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:08.356930 30916 raft_consensus.cc:482] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:08.356962 30916 raft_consensus.cc:3037] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:08.357962 30916 raft_consensus.cc:504] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:08.358342 30916 leader_election.cc:288] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:08.358654 29313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e01c5b97e2d14976812691b78a5143c5" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:08.358768 29313 raft_consensus.cc:3037] T e01c5b97e2d14976812691b78a5143c5 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:08.359699 29313 raft_consensus.cc:2454] T e01c5b97e2d14976812691b78a5143c5 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:08.359926 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e01c5b97e2d14976812691b78a5143c5" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:08.360034 29577 raft_consensus.cc:3037] T e01c5b97e2d14976812691b78a5143c5 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:08.360100 29119 leader_election.cc:302] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:08.360292 30916 raft_consensus.cc:2781] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:08.360440 30916 raft_consensus.cc:686] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:08.360581 30916 consensus_queue.cc:237] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:08.360910 29577 raft_consensus.cc:2454] T e01c5b97e2d14976812691b78a5143c5 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:08.362721 29037 catalog_manager.cc:5484] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:08.418538 30513 raft_consensus.cc:482] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:08.418658 30513 raft_consensus.cc:504] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:08.419030 30513 leader_election.cc:288] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:08.419293 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "791a0a29d67447fcb54e38e848680330" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:08.419384 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "791a0a29d67447fcb54e38e848680330" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:08.419440 29446 raft_consensus.cc:2454] T 791a0a29d67447fcb54e38e848680330 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:08.419498 29577 raft_consensus.cc:2454] T 791a0a29d67447fcb54e38e848680330 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:08.419724 29250 leader_election.cc:302] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:08.419919 30513 raft_consensus.cc:2781] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:08.419972 30513 raft_consensus.cc:482] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:08.420002 30513 raft_consensus.cc:3037] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:08.420908 30513 raft_consensus.cc:504] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:08.421269 30513 leader_election.cc:288] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:08.421494 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "791a0a29d67447fcb54e38e848680330" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:08.421566 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "791a0a29d67447fcb54e38e848680330" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:08.421612 29446 raft_consensus.cc:3037] T 791a0a29d67447fcb54e38e848680330 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:08.421670 29577 raft_consensus.cc:3037] T 791a0a29d67447fcb54e38e848680330 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:08.422561 29446 raft_consensus.cc:2454] T 791a0a29d67447fcb54e38e848680330 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:08.422561 29577 raft_consensus.cc:2454] T 791a0a29d67447fcb54e38e848680330 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:08.422822 29250 leader_election.cc:302] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:08.422986 30513 raft_consensus.cc:2781] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:08.423069 30513 raft_consensus.cc:686] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:08.423195 30513 consensus_queue.cc:237] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:08.424892 29037 catalog_manager.cc:5484] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
W20240502 20:46:08.574378 31201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:08.574652 31201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:08.577845 31201 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_16'
I20240502 20:46:08.662340 29313 raft_consensus.cc:1261] T e01c5b97e2d14976812691b78a5143c5 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:08.663048 30756 consensus_queue.cc:1035] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:08.663897 29575 raft_consensus.cc:1261] T e01c5b97e2d14976812691b78a5143c5 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:08.665498 30916 consensus_queue.cc:1035] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:08.666405 29577 raft_consensus.cc:1261] T 791a0a29d67447fcb54e38e848680330 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:08.666996 29175 raft_consensus.cc:1261] T 1d0d00056f794ed89631137c89569385 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:08.667212 30513 consensus_queue.cc:1035] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:08.668463 30513 consensus_queue.cc:1035] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:08.669245 29444 raft_consensus.cc:1261] T 791a0a29d67447fcb54e38e848680330 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:08.669631 29446 raft_consensus.cc:1261] T 1d0d00056f794ed89631137c89569385 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:08.669843 31232 consensus_queue.cc:1035] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:08.682816 31232 consensus_queue.cc:1035] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:08.693627 30883 mvcc.cc:205] Tried to move back new op lower bound from 7023340620455374848 to 7023340619462696960. Current Snapshot: MvccSnapshot[applied={T|T < 7023340620440285184}]
INSERT report
rows total: 2000
time total: 106.008 ms
time per row: 0.0530038 ms
I20240502 20:46:08.729812 29039 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalance_test_table_17"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:08.735215 29163 tablet_service.cc:1467] Processing CreateTablet for tablet 849c31557dec4043ad36dbddc6811b3f (DEFAULT_TABLE table=rebalance_test_table_17 [id=b1432bc7242c45e38bf75a9c91c6646c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:08.735462 29555 tablet_service.cc:1467] Processing CreateTablet for tablet 849c31557dec4043ad36dbddc6811b3f (DEFAULT_TABLE table=rebalance_test_table_17 [id=b1432bc7242c45e38bf75a9c91c6646c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:08.735563 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 849c31557dec4043ad36dbddc6811b3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:08.735765 29555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 849c31557dec4043ad36dbddc6811b3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:08.735725 29557 tablet_service.cc:1467] Processing CreateTablet for tablet 4c4440d27d9e4070a8e425d26d8a0772 (DEFAULT_TABLE table=rebalance_test_table_17 [id=b1432bc7242c45e38bf75a9c91c6646c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:08.735949 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c4440d27d9e4070a8e425d26d8a0772. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:08.736420 29424 tablet_service.cc:1467] Processing CreateTablet for tablet 849c31557dec4043ad36dbddc6811b3f (DEFAULT_TABLE table=rebalance_test_table_17 [id=b1432bc7242c45e38bf75a9c91c6646c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:08.736675 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 849c31557dec4043ad36dbddc6811b3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:08.737885 31136 tablet_bootstrap.cc:492] T 4c4440d27d9e4070a8e425d26d8a0772 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:08.737977 31246 tablet_bootstrap.cc:492] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:08.738612 31136 tablet_bootstrap.cc:654] T 4c4440d27d9e4070a8e425d26d8a0772 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:08.738746 31138 tablet_bootstrap.cc:492] T 849c31557dec4043ad36dbddc6811b3f P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:08.738961 31246 tablet_bootstrap.cc:654] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:08.738903 29426 tablet_service.cc:1467] Processing CreateTablet for tablet 4c4440d27d9e4070a8e425d26d8a0772 (DEFAULT_TABLE table=rebalance_test_table_17 [id=b1432bc7242c45e38bf75a9c91c6646c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:08.739127 29426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c4440d27d9e4070a8e425d26d8a0772. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:08.739562 31138 tablet_bootstrap.cc:654] T 849c31557dec4043ad36dbddc6811b3f P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:08.740317 29555 tablet_service.cc:1467] Processing CreateTablet for tablet f19fa5a9a0e840c1bc0bc379a1907383 (DEFAULT_TABLE table=rebalance_test_table_17 [id=b1432bc7242c45e38bf75a9c91c6646c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:08.740523 29555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f19fa5a9a0e840c1bc0bc379a1907383. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:08.740669 31136 tablet_bootstrap.cc:492] T 4c4440d27d9e4070a8e425d26d8a0772 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:08.740653 29163 tablet_service.cc:1467] Processing CreateTablet for tablet 4c4440d27d9e4070a8e425d26d8a0772 (DEFAULT_TABLE table=rebalance_test_table_17 [id=b1432bc7242c45e38bf75a9c91c6646c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:08.740741 31136 ts_tablet_manager.cc:1367] T 4c4440d27d9e4070a8e425d26d8a0772 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:08.740883 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c4440d27d9e4070a8e425d26d8a0772. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:08.741181 31136 raft_consensus.cc:348] T 4c4440d27d9e4070a8e425d26d8a0772 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:08.741295 31136 raft_consensus.cc:374] T 4c4440d27d9e4070a8e425d26d8a0772 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:08.741326 31136 raft_consensus.cc:729] T 4c4440d27d9e4070a8e425d26d8a0772 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:08.741300 29164 tablet_service.cc:1467] Processing CreateTablet for tablet f19fa5a9a0e840c1bc0bc379a1907383 (DEFAULT_TABLE table=rebalance_test_table_17 [id=b1432bc7242c45e38bf75a9c91c6646c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:08.741441 31136 consensus_queue.cc:260] T 4c4440d27d9e4070a8e425d26d8a0772 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:08.741513 29164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f19fa5a9a0e840c1bc0bc379a1907383. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:08.741710 31136 ts_tablet_manager.cc:1398] T 4c4440d27d9e4070a8e425d26d8a0772 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:08.741897 31136 tablet_bootstrap.cc:492] T 849c31557dec4043ad36dbddc6811b3f P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:08.742766 31136 tablet_bootstrap.cc:654] T 849c31557dec4043ad36dbddc6811b3f P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:08.742857 31138 tablet_bootstrap.cc:492] T 849c31557dec4043ad36dbddc6811b3f P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:08.742950 31138 ts_tablet_manager.cc:1367] T 849c31557dec4043ad36dbddc6811b3f P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240502 20:46:08.743431 31138 raft_consensus.cc:348] T 849c31557dec4043ad36dbddc6811b3f P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:08.743554 31138 raft_consensus.cc:374] T 849c31557dec4043ad36dbddc6811b3f P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:08.743587 31138 raft_consensus.cc:729] T 849c31557dec4043ad36dbddc6811b3f P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:08.743695 31138 consensus_queue.cc:260] T 849c31557dec4043ad36dbddc6811b3f P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:08.744014 31138 ts_tablet_manager.cc:1398] T 849c31557dec4043ad36dbddc6811b3f P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:08.743957 29295 tablet_service.cc:1467] Processing CreateTablet for tablet f19fa5a9a0e840c1bc0bc379a1907383 (DEFAULT_TABLE table=rebalance_test_table_17 [id=b1432bc7242c45e38bf75a9c91c6646c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:08.744186 31138 tablet_bootstrap.cc:492] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:08.744242 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f19fa5a9a0e840c1bc0bc379a1907383. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:08.745990 31138 tablet_bootstrap.cc:654] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:08.746359 31141 tablet_bootstrap.cc:492] T f19fa5a9a0e840c1bc0bc379a1907383 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:08.747063 31138 tablet_bootstrap.cc:492] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:08.747144 31141 tablet_bootstrap.cc:654] T f19fa5a9a0e840c1bc0bc379a1907383 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:08.747149 31138 ts_tablet_manager.cc:1367] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 20:46:08.747550 31138 raft_consensus.cc:348] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:08.747635 31138 raft_consensus.cc:374] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:08.747664 31138 raft_consensus.cc:729] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:08.747768 31138 consensus_queue.cc:260] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:08.748024 31138 ts_tablet_manager.cc:1398] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:08.750336 31141 tablet_bootstrap.cc:492] T f19fa5a9a0e840c1bc0bc379a1907383 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:08.750337 31136 tablet_bootstrap.cc:492] T 849c31557dec4043ad36dbddc6811b3f P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:08.750422 31136 ts_tablet_manager.cc:1367] T 849c31557dec4043ad36dbddc6811b3f P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 20:46:08.750427 31141 ts_tablet_manager.cc:1367] T f19fa5a9a0e840c1bc0bc379a1907383 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:08.750854 31141 raft_consensus.cc:348] T f19fa5a9a0e840c1bc0bc379a1907383 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:08.750880 31136 raft_consensus.cc:348] T 849c31557dec4043ad36dbddc6811b3f P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:08.750955 31141 raft_consensus.cc:374] T f19fa5a9a0e840c1bc0bc379a1907383 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:08.750962 31136 raft_consensus.cc:374] T 849c31557dec4043ad36dbddc6811b3f P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:08.750985 31141 raft_consensus.cc:729] T f19fa5a9a0e840c1bc0bc379a1907383 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:08.750991 31136 raft_consensus.cc:729] T 849c31557dec4043ad36dbddc6811b3f P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:08.750995 31246 tablet_bootstrap.cc:492] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:08.751060 31246 ts_tablet_manager.cc:1367] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 20:46:08.751098 31136 consensus_queue.cc:260] T 849c31557dec4043ad36dbddc6811b3f P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:08.751098 31141 consensus_queue.cc:260] T f19fa5a9a0e840c1bc0bc379a1907383 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:08.751370 31141 ts_tablet_manager.cc:1398] T f19fa5a9a0e840c1bc0bc379a1907383 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:08.751370 31136 ts_tablet_manager.cc:1398] T 849c31557dec4043ad36dbddc6811b3f P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:08.751590 31136 tablet_bootstrap.cc:492] T f19fa5a9a0e840c1bc0bc379a1907383 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:08.751526 31246 raft_consensus.cc:348] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:08.751626 31246 raft_consensus.cc:374] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:08.751674 31246 raft_consensus.cc:729] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:08.751771 31246 consensus_queue.cc:260] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:08.752455 31136 tablet_bootstrap.cc:654] T f19fa5a9a0e840c1bc0bc379a1907383 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:08.752700 31246 ts_tablet_manager.cc:1398] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:08.752869 31246 tablet_bootstrap.cc:492] T 4c4440d27d9e4070a8e425d26d8a0772 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:08.754063 31246 tablet_bootstrap.cc:654] T 4c4440d27d9e4070a8e425d26d8a0772 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:08.754653 31136 tablet_bootstrap.cc:492] T f19fa5a9a0e840c1bc0bc379a1907383 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:08.754789 31136 ts_tablet_manager.cc:1367] T f19fa5a9a0e840c1bc0bc379a1907383 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:08.755208 31246 tablet_bootstrap.cc:492] T 4c4440d27d9e4070a8e425d26d8a0772 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:08.755291 31246 ts_tablet_manager.cc:1367] T 4c4440d27d9e4070a8e425d26d8a0772 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:08.755313 31136 raft_consensus.cc:348] T f19fa5a9a0e840c1bc0bc379a1907383 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:08.755506 31136 raft_consensus.cc:374] T f19fa5a9a0e840c1bc0bc379a1907383 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:08.755589 31136 raft_consensus.cc:729] T f19fa5a9a0e840c1bc0bc379a1907383 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:08.755844 31136 consensus_queue.cc:260] T f19fa5a9a0e840c1bc0bc379a1907383 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:08.756191 31136 ts_tablet_manager.cc:1398] T f19fa5a9a0e840c1bc0bc379a1907383 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:08.756511 31246 raft_consensus.cc:348] T 4c4440d27d9e4070a8e425d26d8a0772 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:08.756626 31246 raft_consensus.cc:374] T 4c4440d27d9e4070a8e425d26d8a0772 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:08.756682 31246 raft_consensus.cc:729] T 4c4440d27d9e4070a8e425d26d8a0772 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:08.756795 31246 consensus_queue.cc:260] T 4c4440d27d9e4070a8e425d26d8a0772 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:08.757066 31246 ts_tablet_manager.cc:1398] T 4c4440d27d9e4070a8e425d26d8a0772 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:08.757268 31246 tablet_bootstrap.cc:492] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:08.758487 31246 tablet_bootstrap.cc:654] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:08.759477 31246 tablet_bootstrap.cc:492] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:08.759557 31246 ts_tablet_manager.cc:1367] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:08.759917 31246 raft_consensus.cc:348] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:08.760026 31246 raft_consensus.cc:374] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:08.760061 31246 raft_consensus.cc:729] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:08.760167 31246 consensus_queue.cc:260] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:08.760439 31246 ts_tablet_manager.cc:1398] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:08.771206 30892 raft_consensus.cc:482] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:08.771311 30892 raft_consensus.cc:504] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:08.771621 30892 leader_election.cc:288] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:08.771924 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c4440d27d9e4070a8e425d26d8a0772" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:08.771968 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c4440d27d9e4070a8e425d26d8a0772" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:08.772034 29577 raft_consensus.cc:2454] T 4c4440d27d9e4070a8e425d26d8a0772 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:08.772106 29175 raft_consensus.cc:2454] T 4c4440d27d9e4070a8e425d26d8a0772 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:08.772270 29380 leader_election.cc:302] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:08.772465 30892 raft_consensus.cc:2781] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:08.772521 30892 raft_consensus.cc:482] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:08.772563 30892 raft_consensus.cc:3037] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:08.773334 30892 raft_consensus.cc:504] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:08.773639 30892 leader_election.cc:288] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:08.773928 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c4440d27d9e4070a8e425d26d8a0772" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:08.774024 29175 raft_consensus.cc:3037] T 4c4440d27d9e4070a8e425d26d8a0772 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:08.774562 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c4440d27d9e4070a8e425d26d8a0772" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:08.774672 29577 raft_consensus.cc:3037] T 4c4440d27d9e4070a8e425d26d8a0772 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:08.774813 29175 raft_consensus.cc:2454] T 4c4440d27d9e4070a8e425d26d8a0772 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:08.775096 29382 leader_election.cc:302] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:08.775322 30892 raft_consensus.cc:2781] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:08.775393 30892 raft_consensus.cc:686] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:08.775518 30892 consensus_queue.cc:237] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:08.775563 29577 raft_consensus.cc:2454] T 4c4440d27d9e4070a8e425d26d8a0772 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:08.777222 29038 catalog_manager.cc:5484] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:08.802798 31240 raft_consensus.cc:482] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:08.802922 31240 raft_consensus.cc:504] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:08.803378 31240 leader_election.cc:288] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:08.803668 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "849c31557dec4043ad36dbddc6811b3f" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:08.803709 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "849c31557dec4043ad36dbddc6811b3f" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:08.803802 29444 raft_consensus.cc:2454] T 849c31557dec4043ad36dbddc6811b3f P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:08.803876 29575 raft_consensus.cc:2454] T 849c31557dec4043ad36dbddc6811b3f P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:08.804090 29119 leader_election.cc:302] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:08.804297 31240 raft_consensus.cc:2781] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:08.804363 31240 raft_consensus.cc:482] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:08.804388 31240 raft_consensus.cc:3037] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:08.805253 31240 raft_consensus.cc:504] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:08.805555 31240 leader_election.cc:288] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:08.805855 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "849c31557dec4043ad36dbddc6811b3f" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:08.805863 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "849c31557dec4043ad36dbddc6811b3f" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:08.805949 29575 raft_consensus.cc:3037] T 849c31557dec4043ad36dbddc6811b3f P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:08.805954 29444 raft_consensus.cc:3037] T 849c31557dec4043ad36dbddc6811b3f P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:08.806700 29575 raft_consensus.cc:2454] T 849c31557dec4043ad36dbddc6811b3f P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:08.806735 29444 raft_consensus.cc:2454] T 849c31557dec4043ad36dbddc6811b3f P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:08.806947 29118 leader_election.cc:302] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:08.807142 31240 raft_consensus.cc:2781] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:08.807240 31240 raft_consensus.cc:686] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:08.807376 31240 consensus_queue.cc:237] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:08.809180 29038 catalog_manager.cc:5484] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:08.809967 31240 raft_consensus.cc:482] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:08.810063 31240 raft_consensus.cc:504] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:08.810360 31240 leader_election.cc:288] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:08.810724 29313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f19fa5a9a0e840c1bc0bc379a1907383" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:08.810842 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f19fa5a9a0e840c1bc0bc379a1907383" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:08.810869 29313 raft_consensus.cc:2454] T f19fa5a9a0e840c1bc0bc379a1907383 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:08.810945 29575 raft_consensus.cc:2454] T f19fa5a9a0e840c1bc0bc379a1907383 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:08.811196 29119 leader_election.cc:302] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:08.811338 31240 raft_consensus.cc:2781] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:08.811394 31240 raft_consensus.cc:482] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:08.811419 31240 raft_consensus.cc:3037] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:08.812287 31240 raft_consensus.cc:504] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:08.812875 31240 leader_election.cc:288] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:08.812849 29313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f19fa5a9a0e840c1bc0bc379a1907383" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:08.812951 29313 raft_consensus.cc:3037] T f19fa5a9a0e840c1bc0bc379a1907383 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:08.813167 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f19fa5a9a0e840c1bc0bc379a1907383" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:08.813282 29575 raft_consensus.cc:3037] T f19fa5a9a0e840c1bc0bc379a1907383 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:08.813902 29313 raft_consensus.cc:2454] T f19fa5a9a0e840c1bc0bc379a1907383 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:08.814190 29575 raft_consensus.cc:2454] T f19fa5a9a0e840c1bc0bc379a1907383 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:08.814365 29119 leader_election.cc:302] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:08.814545 31240 raft_consensus.cc:2781] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:08.814653 31240 raft_consensus.cc:686] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:08.814774 31240 consensus_queue.cc:237] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:08.816371 29038 catalog_manager.cc:5484] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
W20240502 20:46:08.940378 31259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:08.940626 31259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:08.943818 31259 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_17'
I20240502 20:46:09.018710 29577 raft_consensus.cc:1261] T 4c4440d27d9e4070a8e425d26d8a0772 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:09.019104 29175 raft_consensus.cc:1261] T 4c4440d27d9e4070a8e425d26d8a0772 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:09.019207 31016 consensus_queue.cc:1035] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:09.024212 31016 consensus_queue.cc:1035] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:09.024966 29575 raft_consensus.cc:1261] T f19fa5a9a0e840c1bc0bc379a1907383 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:09.025508 30756 consensus_queue.cc:1035] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:09.037073 29314 raft_consensus.cc:1261] T f19fa5a9a0e840c1bc0bc379a1907383 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:09.037478 30756 consensus_queue.cc:1035] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:09.040035 29574 raft_consensus.cc:1261] T 849c31557dec4043ad36dbddc6811b3f P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:09.041693 30756 consensus_queue.cc:1035] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:09.042053 29444 raft_consensus.cc:1261] T 849c31557dec4043ad36dbddc6811b3f P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:09.042440 30756 consensus_queue.cc:1035] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:09.055076 30895 mvcc.cc:205] Tried to move back new op lower bound from 7023340621896097792 to 7023340620905676800. Current Snapshot: MvccSnapshot[applied={T|T < 7023340621896097792}]
I20240502 20:46:09.062541 31184 mvcc.cc:205] Tried to move back new op lower bound from 7023340621979660288 to 7023340621036322816. Current Snapshot: MvccSnapshot[applied={T|T < 7023340621979660288}]
INSERT report
rows total: 2000
time total: 111.762 ms
time per row: 0.0558809 ms
I20240502 20:46:09.096973 29039 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalance_test_table_18"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:09.102289 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 62f3301937ba4471afcc327be05486cf (DEFAULT_TABLE table=rebalance_test_table_18 [id=8dca593309e7400190f93e57aecd5c8e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:09.102316 29426 tablet_service.cc:1467] Processing CreateTablet for tablet 62f3301937ba4471afcc327be05486cf (DEFAULT_TABLE table=rebalance_test_table_18 [id=8dca593309e7400190f93e57aecd5c8e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:09.102648 29426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62f3301937ba4471afcc327be05486cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:09.102651 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62f3301937ba4471afcc327be05486cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:09.102979 29555 tablet_service.cc:1467] Processing CreateTablet for tablet 7e6eb9c58e764ab0be9fb330de4c4ddb (DEFAULT_TABLE table=rebalance_test_table_18 [id=8dca593309e7400190f93e57aecd5c8e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:09.103225 29555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e6eb9c58e764ab0be9fb330de4c4ddb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:09.104385 31138 tablet_bootstrap.cc:492] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:09.104620 29164 tablet_service.cc:1467] Processing CreateTablet for tablet 62f3301937ba4471afcc327be05486cf (DEFAULT_TABLE table=rebalance_test_table_18 [id=8dca593309e7400190f93e57aecd5c8e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:09.104809 31141 tablet_bootstrap.cc:492] T 62f3301937ba4471afcc327be05486cf P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:09.104877 29164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62f3301937ba4471afcc327be05486cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:09.105235 31138 tablet_bootstrap.cc:654] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:09.105460 29163 tablet_service.cc:1467] Processing CreateTablet for tablet 5fd8d498911640cfb747d981c3e73f06 (DEFAULT_TABLE table=rebalance_test_table_18 [id=8dca593309e7400190f93e57aecd5c8e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:09.105614 31141 tablet_bootstrap.cc:654] T 62f3301937ba4471afcc327be05486cf P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:09.105681 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fd8d498911640cfb747d981c3e73f06. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:09.106146 31136 tablet_bootstrap.cc:492] T 7e6eb9c58e764ab0be9fb330de4c4ddb P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:09.106590 31141 tablet_bootstrap.cc:492] T 62f3301937ba4471afcc327be05486cf P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:09.106676 31141 ts_tablet_manager.cc:1367] T 62f3301937ba4471afcc327be05486cf P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:09.106979 31136 tablet_bootstrap.cc:654] T 7e6eb9c58e764ab0be9fb330de4c4ddb P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:09.107103 31141 raft_consensus.cc:348] T 62f3301937ba4471afcc327be05486cf P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:09.107215 31141 raft_consensus.cc:374] T 62f3301937ba4471afcc327be05486cf P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:09.107245 31141 raft_consensus.cc:729] T 62f3301937ba4471afcc327be05486cf P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:09.107378 31141 consensus_queue.cc:260] T 62f3301937ba4471afcc327be05486cf P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:09.107661 31141 ts_tablet_manager.cc:1398] T 62f3301937ba4471afcc327be05486cf P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:09.107699 31246 tablet_bootstrap.cc:492] T 62f3301937ba4471afcc327be05486cf P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:09.108543 31246 tablet_bootstrap.cc:654] T 62f3301937ba4471afcc327be05486cf P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:09.108901 31138 tablet_bootstrap.cc:492] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:09.108975 31138 ts_tablet_manager.cc:1367] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240502 20:46:09.109439 31138 raft_consensus.cc:348] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:09.109541 31138 raft_consensus.cc:374] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:09.109573 31138 raft_consensus.cc:729] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:09.109683 31138 consensus_queue.cc:260] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:09.109876 29426 tablet_service.cc:1467] Processing CreateTablet for tablet 7e6eb9c58e764ab0be9fb330de4c4ddb (DEFAULT_TABLE table=rebalance_test_table_18 [id=8dca593309e7400190f93e57aecd5c8e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:09.109966 31138 ts_tablet_manager.cc:1398] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:09.110066 31246 tablet_bootstrap.cc:492] T 62f3301937ba4471afcc327be05486cf P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:09.110096 29426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e6eb9c58e764ab0be9fb330de4c4ddb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:09.110139 31246 ts_tablet_manager.cc:1367] T 62f3301937ba4471afcc327be05486cf P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:09.110303 29424 tablet_service.cc:1467] Processing CreateTablet for tablet 5fd8d498911640cfb747d981c3e73f06 (DEFAULT_TABLE table=rebalance_test_table_18 [id=8dca593309e7400190f93e57aecd5c8e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:09.110524 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fd8d498911640cfb747d981c3e73f06. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:09.110601 31246 raft_consensus.cc:348] T 62f3301937ba4471afcc327be05486cf P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:09.110699 31246 raft_consensus.cc:374] T 62f3301937ba4471afcc327be05486cf P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:09.110728 31246 raft_consensus.cc:729] T 62f3301937ba4471afcc327be05486cf P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:09.110839 31246 consensus_queue.cc:260] T 62f3301937ba4471afcc327be05486cf P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:09.111088 31246 ts_tablet_manager.cc:1398] T 62f3301937ba4471afcc327be05486cf P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:09.111244 31246 tablet_bootstrap.cc:492] T 5fd8d498911640cfb747d981c3e73f06 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:09.111485 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 7e6eb9c58e764ab0be9fb330de4c4ddb (DEFAULT_TABLE table=rebalance_test_table_18 [id=8dca593309e7400190f93e57aecd5c8e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:09.111706 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e6eb9c58e764ab0be9fb330de4c4ddb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:09.112023 31246 tablet_bootstrap.cc:654] T 5fd8d498911640cfb747d981c3e73f06 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:09.113698 31136 tablet_bootstrap.cc:492] T 7e6eb9c58e764ab0be9fb330de4c4ddb P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:09.113780 31136 ts_tablet_manager.cc:1367] T 7e6eb9c58e764ab0be9fb330de4c4ddb P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240502 20:46:09.113786 29294 tablet_service.cc:1467] Processing CreateTablet for tablet 5fd8d498911640cfb747d981c3e73f06 (DEFAULT_TABLE table=rebalance_test_table_18 [id=8dca593309e7400190f93e57aecd5c8e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:09.113991 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fd8d498911640cfb747d981c3e73f06. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:09.114249 31136 raft_consensus.cc:348] T 7e6eb9c58e764ab0be9fb330de4c4ddb P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:09.114344 31136 raft_consensus.cc:374] T 7e6eb9c58e764ab0be9fb330de4c4ddb P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:09.114369 31136 raft_consensus.cc:729] T 7e6eb9c58e764ab0be9fb330de4c4ddb P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:09.114599 31136 consensus_queue.cc:260] T 7e6eb9c58e764ab0be9fb330de4c4ddb P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:09.114868 31136 ts_tablet_manager.cc:1398] T 7e6eb9c58e764ab0be9fb330de4c4ddb P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:09.115621 31138 tablet_bootstrap.cc:492] T 7e6eb9c58e764ab0be9fb330de4c4ddb P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:09.116427 31141 tablet_bootstrap.cc:492] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:09.116565 31138 tablet_bootstrap.cc:654] T 7e6eb9c58e764ab0be9fb330de4c4ddb P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:09.117233 31141 tablet_bootstrap.cc:654] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:09.117909 31138 tablet_bootstrap.cc:492] T 7e6eb9c58e764ab0be9fb330de4c4ddb P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:09.118042 31138 ts_tablet_manager.cc:1367] T 7e6eb9c58e764ab0be9fb330de4c4ddb P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 20:46:09.118564 31138 raft_consensus.cc:348] T 7e6eb9c58e764ab0be9fb330de4c4ddb P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:09.118723 31138 raft_consensus.cc:374] T 7e6eb9c58e764ab0be9fb330de4c4ddb P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:09.118804 31138 raft_consensus.cc:729] T 7e6eb9c58e764ab0be9fb330de4c4ddb P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:09.119011 31246 tablet_bootstrap.cc:492] T 5fd8d498911640cfb747d981c3e73f06 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:09.119000 31138 consensus_queue.cc:260] T 7e6eb9c58e764ab0be9fb330de4c4ddb P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:09.119088 31246 ts_tablet_manager.cc:1367] T 5fd8d498911640cfb747d981c3e73f06 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 20:46:09.119336 31138 ts_tablet_manager.cc:1398] T 7e6eb9c58e764ab0be9fb330de4c4ddb P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:09.119493 31138 tablet_bootstrap.cc:492] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:09.119448 31246 raft_consensus.cc:348] T 5fd8d498911640cfb747d981c3e73f06 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:09.119572 31246 raft_consensus.cc:374] T 5fd8d498911640cfb747d981c3e73f06 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:09.119601 31246 raft_consensus.cc:729] T 5fd8d498911640cfb747d981c3e73f06 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:09.119721 31246 consensus_queue.cc:260] T 5fd8d498911640cfb747d981c3e73f06 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:09.119989 31246 ts_tablet_manager.cc:1398] T 5fd8d498911640cfb747d981c3e73f06 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:09.120801 31141 tablet_bootstrap.cc:492] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:09.120885 31141 ts_tablet_manager.cc:1367] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:09.121311 31141 raft_consensus.cc:348] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:09.121421 31141 raft_consensus.cc:374] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:09.121460 31141 raft_consensus.cc:729] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:09.121384 31138 tablet_bootstrap.cc:654] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:09.121572 31141 consensus_queue.cc:260] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:09.121840 31141 ts_tablet_manager.cc:1398] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:09.122002 31141 tablet_bootstrap.cc:492] T 5fd8d498911640cfb747d981c3e73f06 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:09.122895 31141 tablet_bootstrap.cc:654] T 5fd8d498911640cfb747d981c3e73f06 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:09.123441 31138 tablet_bootstrap.cc:492] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:09.123518 31138 ts_tablet_manager.cc:1367] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:09.123921 31138 raft_consensus.cc:348] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:09.124002 31138 raft_consensus.cc:374] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:09.124029 31138 raft_consensus.cc:729] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:09.124145 31138 consensus_queue.cc:260] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:09.124423 31138 ts_tablet_manager.cc:1398] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:09.124992 31141 tablet_bootstrap.cc:492] T 5fd8d498911640cfb747d981c3e73f06 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:09.125070 31141 ts_tablet_manager.cc:1367] T 5fd8d498911640cfb747d981c3e73f06 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 20:46:09.125468 31141 raft_consensus.cc:348] T 5fd8d498911640cfb747d981c3e73f06 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:09.125562 31141 raft_consensus.cc:374] T 5fd8d498911640cfb747d981c3e73f06 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:09.125586 31141 raft_consensus.cc:729] T 5fd8d498911640cfb747d981c3e73f06 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:09.125710 31141 consensus_queue.cc:260] T 5fd8d498911640cfb747d981c3e73f06 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:09.125988 31141 ts_tablet_manager.cc:1398] T 5fd8d498911640cfb747d981c3e73f06 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:09.133487 30892 raft_consensus.cc:482] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:09.133576 30892 raft_consensus.cc:504] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:09.133909 30892 leader_election.cc:288] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:09.134205 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62f3301937ba4471afcc327be05486cf" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:09.134274 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62f3301937ba4471afcc327be05486cf" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:09.134323 29314 raft_consensus.cc:2454] T 62f3301937ba4471afcc327be05486cf P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:09.134413 29175 raft_consensus.cc:2454] T 62f3301937ba4471afcc327be05486cf P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:09.134595 29381 leader_election.cc:302] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:09.134754 30892 raft_consensus.cc:2781] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:09.134810 30892 raft_consensus.cc:482] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:09.134845 30892 raft_consensus.cc:3037] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:09.135614 30892 raft_consensus.cc:504] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:09.135907 30892 leader_election.cc:288] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:09.136212 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62f3301937ba4471afcc327be05486cf" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:09.136217 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62f3301937ba4471afcc327be05486cf" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:09.136322 29314 raft_consensus.cc:3037] T 62f3301937ba4471afcc327be05486cf P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:09.136322 29175 raft_consensus.cc:3037] T 62f3301937ba4471afcc327be05486cf P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:09.137226 29314 raft_consensus.cc:2454] T 62f3301937ba4471afcc327be05486cf P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:09.137226 29175 raft_consensus.cc:2454] T 62f3301937ba4471afcc327be05486cf P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:09.137483 29382 leader_election.cc:302] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:09.137636 30892 raft_consensus.cc:2781] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:09.137723 30892 raft_consensus.cc:686] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:09.137866 30892 consensus_queue.cc:237] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:09.139736 29037 catalog_manager.cc:5484] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:09.278270 30892 raft_consensus.cc:482] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:09.278399 30892 raft_consensus.cc:504] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:09.278839 30892 leader_election.cc:288] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:09.279201 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fd8d498911640cfb747d981c3e73f06" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:09.279225 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fd8d498911640cfb747d981c3e73f06" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:09.279341 29175 raft_consensus.cc:2454] T 5fd8d498911640cfb747d981c3e73f06 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:09.279353 29314 raft_consensus.cc:2454] T 5fd8d498911640cfb747d981c3e73f06 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:09.279624 29382 leader_election.cc:302] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:09.279785 30892 raft_consensus.cc:2781] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:09.279845 30892 raft_consensus.cc:482] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:09.279871 30892 raft_consensus.cc:3037] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:09.280673 30892 raft_consensus.cc:504] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:09.281006 30892 leader_election.cc:288] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:09.281309 31231 raft_consensus.cc:482] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:09.281320 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fd8d498911640cfb747d981c3e73f06" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:09.281349 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fd8d498911640cfb747d981c3e73f06" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:09.281457 29175 raft_consensus.cc:3037] T 5fd8d498911640cfb747d981c3e73f06 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:09.281425 29314 raft_consensus.cc:3037] T 5fd8d498911640cfb747d981c3e73f06 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:09.281412 31231 raft_consensus.cc:504] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:09.281849 31231 leader_election.cc:288] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:09.282213 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e6eb9c58e764ab0be9fb330de4c4ddb" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:09.282351 29444 raft_consensus.cc:2454] T 7e6eb9c58e764ab0be9fb330de4c4ddb P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:09.282411 29175 raft_consensus.cc:2454] T 5fd8d498911640cfb747d981c3e73f06 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:09.282476 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e6eb9c58e764ab0be9fb330de4c4ddb" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:09.282550 29314 raft_consensus.cc:2454] T 5fd8d498911640cfb747d981c3e73f06 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:09.282605 29577 raft_consensus.cc:2454] T 7e6eb9c58e764ab0be9fb330de4c4ddb P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:09.282791 29381 leader_election.cc:302] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:09.282917 29249 leader_election.cc:302] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:09.282948 30892 raft_consensus.cc:2781] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:09.283083 31231 raft_consensus.cc:2781] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:09.283108 30892 raft_consensus.cc:686] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:09.283133 31231 raft_consensus.cc:482] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:09.283175 31231 raft_consensus.cc:3037] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:09.283248 30892 consensus_queue.cc:237] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:09.284071 31231 raft_consensus.cc:504] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:09.284386 31231 leader_election.cc:288] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:09.284689 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e6eb9c58e764ab0be9fb330de4c4ddb" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:09.284770 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e6eb9c58e764ab0be9fb330de4c4ddb" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:09.284796 29444 raft_consensus.cc:3037] T 7e6eb9c58e764ab0be9fb330de4c4ddb P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:09.284868 29577 raft_consensus.cc:3037] T 7e6eb9c58e764ab0be9fb330de4c4ddb P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:09.285704 29444 raft_consensus.cc:2454] T 7e6eb9c58e764ab0be9fb330de4c4ddb P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:09.285688 29577 raft_consensus.cc:2454] T 7e6eb9c58e764ab0be9fb330de4c4ddb P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:09.285967 29250 leader_election.cc:302] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:09.286186 31231 raft_consensus.cc:2781] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:09.286296 31231 raft_consensus.cc:686] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:09.286612 31231 consensus_queue.cc:237] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:09.286614 29037 catalog_manager.cc:5484] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:09.288412 29033 catalog_manager.cc:5484] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
W20240502 20:46:09.415221 31306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:09.415457 31306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:09.418566 31306 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_18'
I20240502 20:46:09.514879 29182 raft_consensus.cc:1261] T 5fd8d498911640cfb747d981c3e73f06 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:09.514879 29314 raft_consensus.cc:1261] T 5fd8d498911640cfb747d981c3e73f06 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:09.515312 31016 consensus_queue.cc:1035] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:09.515502 30892 consensus_queue.cc:1035] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:09.519219 29444 raft_consensus.cc:1261] T 7e6eb9c58e764ab0be9fb330de4c4ddb P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:09.521238 29577 raft_consensus.cc:1261] T 7e6eb9c58e764ab0be9fb330de4c4ddb P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:09.521294 31328 consensus_queue.cc:1035] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:09.521957 31328 consensus_queue.cc:1035] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:09.542335 29175 raft_consensus.cc:1261] T 62f3301937ba4471afcc327be05486cf P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:09.542904 30892 consensus_queue.cc:1035] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:09.543103 29314 raft_consensus.cc:1261] T 62f3301937ba4471afcc327be05486cf P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:09.545094 31016 consensus_queue.cc:1035] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 127.298 ms
time per row: 0.0636488 ms
I20240502 20:46:09.586628 29034 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalance_test_table_19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:09.592011 29294 tablet_service.cc:1467] Processing CreateTablet for tablet b28fe6bccf4f48a6a5456e6ead5cc920 (DEFAULT_TABLE table=rebalance_test_table_19 [id=5fd0afffa41641b9b8de6545456ff39a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:09.592083 29555 tablet_service.cc:1467] Processing CreateTablet for tablet b28fe6bccf4f48a6a5456e6ead5cc920 (DEFAULT_TABLE table=rebalance_test_table_19 [id=5fd0afffa41641b9b8de6545456ff39a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:09.592378 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b28fe6bccf4f48a6a5456e6ead5cc920. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:09.592406 29555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b28fe6bccf4f48a6a5456e6ead5cc920. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:09.592365 29424 tablet_service.cc:1467] Processing CreateTablet for tablet c18f9efa9c2043f6b2fad95ad9fd8296 (DEFAULT_TABLE table=rebalance_test_table_19 [id=5fd0afffa41641b9b8de6545456ff39a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:09.592612 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c18f9efa9c2043f6b2fad95ad9fd8296. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:09.593930 29295 tablet_service.cc:1467] Processing CreateTablet for tablet c18f9efa9c2043f6b2fad95ad9fd8296 (DEFAULT_TABLE table=rebalance_test_table_19 [id=5fd0afffa41641b9b8de6545456ff39a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:09.594151 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c18f9efa9c2043f6b2fad95ad9fd8296. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:09.594601 31136 tablet_bootstrap.cc:492] T b28fe6bccf4f48a6a5456e6ead5cc920 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:09.594682 31138 tablet_bootstrap.cc:492] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:09.594695 31141 tablet_bootstrap.cc:492] T b28fe6bccf4f48a6a5456e6ead5cc920 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:09.595057 29426 tablet_service.cc:1467] Processing CreateTablet for tablet 8c21a6762eaa4ba4ac1c91a42fb37849 (DEFAULT_TABLE table=rebalance_test_table_19 [id=5fd0afffa41641b9b8de6545456ff39a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:09.595281 29426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c21a6762eaa4ba4ac1c91a42fb37849. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:09.595494 31141 tablet_bootstrap.cc:654] T b28fe6bccf4f48a6a5456e6ead5cc920 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:09.595495 31136 tablet_bootstrap.cc:654] T b28fe6bccf4f48a6a5456e6ead5cc920 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:09.595541 31138 tablet_bootstrap.cc:654] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:09.596210 29163 tablet_service.cc:1467] Processing CreateTablet for tablet b28fe6bccf4f48a6a5456e6ead5cc920 (DEFAULT_TABLE table=rebalance_test_table_19 [id=5fd0afffa41641b9b8de6545456ff39a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:09.596298 29164 tablet_service.cc:1467] Processing CreateTablet for tablet 8c21a6762eaa4ba4ac1c91a42fb37849 (DEFAULT_TABLE table=rebalance_test_table_19 [id=5fd0afffa41641b9b8de6545456ff39a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:09.596427 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b28fe6bccf4f48a6a5456e6ead5cc920. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:09.596527 29164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c21a6762eaa4ba4ac1c91a42fb37849. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:09.598273 31141 tablet_bootstrap.cc:492] T b28fe6bccf4f48a6a5456e6ead5cc920 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:09.598345 31246 tablet_bootstrap.cc:492] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:09.598357 31141 ts_tablet_manager.cc:1367] T b28fe6bccf4f48a6a5456e6ead5cc920 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240502 20:46:09.598812 31141 raft_consensus.cc:348] T b28fe6bccf4f48a6a5456e6ead5cc920 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:09.598917 31141 raft_consensus.cc:374] T b28fe6bccf4f48a6a5456e6ead5cc920 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:09.598944 31141 raft_consensus.cc:729] T b28fe6bccf4f48a6a5456e6ead5cc920 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:09.599056 31141 consensus_queue.cc:260] T b28fe6bccf4f48a6a5456e6ead5cc920 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:09.599229 31246 tablet_bootstrap.cc:654] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:09.599303 31138 tablet_bootstrap.cc:492] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:09.599346 31141 ts_tablet_manager.cc:1398] T b28fe6bccf4f48a6a5456e6ead5cc920 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:09.599376 31138 ts_tablet_manager.cc:1367] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 20:46:09.599680 29557 tablet_service.cc:1467] Processing CreateTablet for tablet 8c21a6762eaa4ba4ac1c91a42fb37849 (DEFAULT_TABLE table=rebalance_test_table_19 [id=5fd0afffa41641b9b8de6545456ff39a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:09.599826 31138 raft_consensus.cc:348] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:09.599915 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c21a6762eaa4ba4ac1c91a42fb37849. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:09.599926 31138 raft_consensus.cc:374] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:09.599957 31138 raft_consensus.cc:729] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:09.600061 29555 tablet_service.cc:1467] Processing CreateTablet for tablet c18f9efa9c2043f6b2fad95ad9fd8296 (DEFAULT_TABLE table=rebalance_test_table_19 [id=5fd0afffa41641b9b8de6545456ff39a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:09.600072 31138 consensus_queue.cc:260] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:09.600284 29555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c18f9efa9c2043f6b2fad95ad9fd8296. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:09.600343 31138 ts_tablet_manager.cc:1398] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:09.600665 31138 tablet_bootstrap.cc:492] T 8c21a6762eaa4ba4ac1c91a42fb37849 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:09.601861 31136 tablet_bootstrap.cc:492] T b28fe6bccf4f48a6a5456e6ead5cc920 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:09.601943 31136 ts_tablet_manager.cc:1367] T b28fe6bccf4f48a6a5456e6ead5cc920 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 20:46:09.602121 31138 tablet_bootstrap.cc:654] T 8c21a6762eaa4ba4ac1c91a42fb37849 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:09.602322 31136 raft_consensus.cc:348] T b28fe6bccf4f48a6a5456e6ead5cc920 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:09.602418 31136 raft_consensus.cc:374] T b28fe6bccf4f48a6a5456e6ead5cc920 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:09.602447 31136 raft_consensus.cc:729] T b28fe6bccf4f48a6a5456e6ead5cc920 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:09.602702 31136 consensus_queue.cc:260] T b28fe6bccf4f48a6a5456e6ead5cc920 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:09.603000 31141 tablet_bootstrap.cc:492] T c18f9efa9c2043f6b2fad95ad9fd8296 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:09.602993 31136 ts_tablet_manager.cc:1398] T b28fe6bccf4f48a6a5456e6ead5cc920 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:09.603708 31136 tablet_bootstrap.cc:492] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:09.604513 31136 tablet_bootstrap.cc:654] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:09.605048 31141 tablet_bootstrap.cc:654] T c18f9efa9c2043f6b2fad95ad9fd8296 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:09.606968 31136 tablet_bootstrap.cc:492] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:09.607048 31136 ts_tablet_manager.cc:1367] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:09.607386 31246 tablet_bootstrap.cc:492] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:09.607465 31246 ts_tablet_manager.cc:1367] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240502 20:46:09.607450 31136 raft_consensus.cc:348] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:09.607537 31136 raft_consensus.cc:374] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:09.607568 31136 raft_consensus.cc:729] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:09.607636 31141 tablet_bootstrap.cc:492] T c18f9efa9c2043f6b2fad95ad9fd8296 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:09.607705 31141 ts_tablet_manager.cc:1367] T c18f9efa9c2043f6b2fad95ad9fd8296 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 20:46:09.607683 31136 consensus_queue.cc:260] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:09.607937 31136 ts_tablet_manager.cc:1398] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:09.608074 31136 tablet_bootstrap.cc:492] T c18f9efa9c2043f6b2fad95ad9fd8296 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:09.608070 31246 raft_consensus.cc:348] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:09.608115 31141 raft_consensus.cc:348] T c18f9efa9c2043f6b2fad95ad9fd8296 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:09.608186 31246 raft_consensus.cc:374] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:09.608219 31141 raft_consensus.cc:374] T c18f9efa9c2043f6b2fad95ad9fd8296 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:09.608215 31246 raft_consensus.cc:729] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:09.608248 31141 raft_consensus.cc:729] T c18f9efa9c2043f6b2fad95ad9fd8296 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:09.608330 31246 consensus_queue.cc:260] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:09.608361 31141 consensus_queue.cc:260] T c18f9efa9c2043f6b2fad95ad9fd8296 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:09.608592 31246 ts_tablet_manager.cc:1398] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:09.608626 31141 ts_tablet_manager.cc:1398] T c18f9efa9c2043f6b2fad95ad9fd8296 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:09.609059 31136 tablet_bootstrap.cc:654] T c18f9efa9c2043f6b2fad95ad9fd8296 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:09.611097 31136 tablet_bootstrap.cc:492] T c18f9efa9c2043f6b2fad95ad9fd8296 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:09.611174 31136 ts_tablet_manager.cc:1367] T c18f9efa9c2043f6b2fad95ad9fd8296 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:09.611516 31136 raft_consensus.cc:348] T c18f9efa9c2043f6b2fad95ad9fd8296 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:09.611609 31136 raft_consensus.cc:374] T c18f9efa9c2043f6b2fad95ad9fd8296 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:09.611650 31136 raft_consensus.cc:729] T c18f9efa9c2043f6b2fad95ad9fd8296 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:09.611752 31136 consensus_queue.cc:260] T c18f9efa9c2043f6b2fad95ad9fd8296 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:09.612003 31136 ts_tablet_manager.cc:1398] T c18f9efa9c2043f6b2fad95ad9fd8296 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:09.612304 31246 tablet_bootstrap.cc:492] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:09.613169 31246 tablet_bootstrap.cc:654] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:09.614086 31138 tablet_bootstrap.cc:492] T 8c21a6762eaa4ba4ac1c91a42fb37849 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:09.614182 31138 ts_tablet_manager.cc:1367] T 8c21a6762eaa4ba4ac1c91a42fb37849 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 20:46:09.614846 31138 raft_consensus.cc:348] T 8c21a6762eaa4ba4ac1c91a42fb37849 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:09.614965 31138 raft_consensus.cc:374] T 8c21a6762eaa4ba4ac1c91a42fb37849 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:09.615007 31138 raft_consensus.cc:729] T 8c21a6762eaa4ba4ac1c91a42fb37849 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:09.615123 31138 consensus_queue.cc:260] T 8c21a6762eaa4ba4ac1c91a42fb37849 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:09.615360 31138 ts_tablet_manager.cc:1398] T 8c21a6762eaa4ba4ac1c91a42fb37849 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:09.619139 31246 tablet_bootstrap.cc:492] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:09.619216 31246 ts_tablet_manager.cc:1367] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 20:46:09.619576 31246 raft_consensus.cc:348] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:09.619666 31246 raft_consensus.cc:374] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:09.619688 31246 raft_consensus.cc:729] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:09.619796 31246 consensus_queue.cc:260] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:09.620043 31246 ts_tablet_manager.cc:1398] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:09.637219 30756 raft_consensus.cc:482] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:09.637360 30756 raft_consensus.cc:504] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:09.637773 30756 leader_election.cc:288] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:09.638056 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b28fe6bccf4f48a6a5456e6ead5cc920" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:09.638111 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b28fe6bccf4f48a6a5456e6ead5cc920" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:09.638182 29314 raft_consensus.cc:2454] T b28fe6bccf4f48a6a5456e6ead5cc920 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:09.638254 29577 raft_consensus.cc:2454] T b28fe6bccf4f48a6a5456e6ead5cc920 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:09.638507 29119 leader_election.cc:302] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:09.638680 30756 raft_consensus.cc:2781] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:09.638747 30756 raft_consensus.cc:482] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:09.638794 30756 raft_consensus.cc:3037] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:09.639593 30756 raft_consensus.cc:504] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:09.639919 30756 leader_election.cc:288] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:09.640166 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b28fe6bccf4f48a6a5456e6ead5cc920" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:09.640237 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b28fe6bccf4f48a6a5456e6ead5cc920" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:09.640300 29314 raft_consensus.cc:3037] T b28fe6bccf4f48a6a5456e6ead5cc920 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:09.640347 29577 raft_consensus.cc:3037] T b28fe6bccf4f48a6a5456e6ead5cc920 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:09.641247 29314 raft_consensus.cc:2454] T b28fe6bccf4f48a6a5456e6ead5cc920 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:09.641243 29577 raft_consensus.cc:2454] T b28fe6bccf4f48a6a5456e6ead5cc920 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:09.641542 29118 leader_election.cc:302] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:09.641701 30756 raft_consensus.cc:2781] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:09.641809 30756 raft_consensus.cc:686] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:09.641958 30756 consensus_queue.cc:237] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:09.643939 29034 catalog_manager.cc:5484] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:09.721606 30892 raft_consensus.cc:482] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:09.721741 30892 raft_consensus.cc:504] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:09.722131 30892 leader_election.cc:288] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:09.722498 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c18f9efa9c2043f6b2fad95ad9fd8296" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:09.722532 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c18f9efa9c2043f6b2fad95ad9fd8296" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:09.722637 29577 raft_consensus.cc:2454] T c18f9efa9c2043f6b2fad95ad9fd8296 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:09.722671 29314 raft_consensus.cc:2454] T c18f9efa9c2043f6b2fad95ad9fd8296 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:09.722965 29381 leader_election.cc:302] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:09.723124 30892 raft_consensus.cc:2781] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:09.723183 30892 raft_consensus.cc:482] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:09.723227 30892 raft_consensus.cc:3037] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:09.724009 30892 raft_consensus.cc:504] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:09.724326 30892 leader_election.cc:288] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:09.724620 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c18f9efa9c2043f6b2fad95ad9fd8296" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:09.724628 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c18f9efa9c2043f6b2fad95ad9fd8296" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:09.724715 29314 raft_consensus.cc:3037] T c18f9efa9c2043f6b2fad95ad9fd8296 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:09.724725 29577 raft_consensus.cc:3037] T c18f9efa9c2043f6b2fad95ad9fd8296 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:09.725487 29314 raft_consensus.cc:2454] T c18f9efa9c2043f6b2fad95ad9fd8296 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:09.725657 29577 raft_consensus.cc:2454] T c18f9efa9c2043f6b2fad95ad9fd8296 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:09.725769 29381 leader_election.cc:302] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:09.725903 30892 raft_consensus.cc:2781] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:09.725982 30892 raft_consensus.cc:686] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:09.726112 30892 consensus_queue.cc:237] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:09.728197 29038 catalog_manager.cc:5484] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:09.849370 30756 raft_consensus.cc:482] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:09.849481 30756 raft_consensus.cc:504] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:09.849829 30756 leader_election.cc:288] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:09.850178 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c21a6762eaa4ba4ac1c91a42fb37849" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:09.850173 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c21a6762eaa4ba4ac1c91a42fb37849" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:09.850318 29577 raft_consensus.cc:2454] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:09.850318 29444 raft_consensus.cc:2454] T 8c21a6762eaa4ba4ac1c91a42fb37849 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:09.850651 29118 leader_election.cc:302] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:09.850966 30756 raft_consensus.cc:2781] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:09.851022 30756 raft_consensus.cc:482] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:09.851058 30756 raft_consensus.cc:3037] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:09.852123 30756 raft_consensus.cc:504] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:09.852735 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c21a6762eaa4ba4ac1c91a42fb37849" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:09.852762 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c21a6762eaa4ba4ac1c91a42fb37849" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:09.852843 29577 raft_consensus.cc:3037] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:09.852878 29444 raft_consensus.cc:3037] T 8c21a6762eaa4ba4ac1c91a42fb37849 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:09.853545 29577 raft_consensus.cc:2454] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:09.852507 30756 leader_election.cc:288] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:09.853801 29118 leader_election.cc:302] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:09.853825 29444 raft_consensus.cc:2454] T 8c21a6762eaa4ba4ac1c91a42fb37849 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:09.853938 30756 raft_consensus.cc:2781] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:09.854087 30756 raft_consensus.cc:686] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:09.854226 30756 consensus_queue.cc:237] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:09.856195 29038 catalog_manager.cc:5484] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
W20240502 20:46:10.011245 31355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:10.011566 31355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:10.015908 31355 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_19'
I20240502 20:46:10.047555 30756 consensus_queue.cc:1035] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:10.051491 30916 consensus_queue.cc:1035] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:10.117116 29444 raft_consensus.cc:1261] T 8c21a6762eaa4ba4ac1c91a42fb37849 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:10.117319 29574 raft_consensus.cc:1261] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:10.117830 29313 raft_consensus.cc:1261] T c18f9efa9c2043f6b2fad95ad9fd8296 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:10.117874 30916 consensus_queue.cc:1035] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:10.118175 30916 consensus_queue.cc:1035] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:10.120119 31016 consensus_queue.cc:1035] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:10.139796 29574 raft_consensus.cc:1261] T c18f9efa9c2043f6b2fad95ad9fd8296 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:10.140260 30892 consensus_queue.cc:1035] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 121.571 ms
time per row: 0.0607854 ms
I20240502 20:46:10.187142 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.69:43887
--local_ip_for_outbound_sockets=127.22.45.69
--tserver_master_addrs=127.22.45.126:34109
--webserver_port=46521
--webserver_interface=127.22.45.69
--builtin_ntp_servers=127.22.45.84:45597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 20:46:10.299962 31413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:10.300171 31413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:46:10.300212 31413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:10.303457 31413 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:46:10.303526 31413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:46:10.303596 31413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.69
I20240502 20:46:10.307250 31413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:45597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.69:43887
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.22.45.69
--webserver_port=46521
--tserver_master_addrs=127.22.45.126:34109
--never_fsync=true
--heap_profile_path=/tmp/kudu.31413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.69
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:46:10.308341 31413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:46:10.309504 31413 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:46:10.317818 31418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:10.318222 31419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:10.318595 31421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:46:10.319073 31413 server_base.cc:1034] running on GCE node
I20240502 20:46:10.319442 31413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:46:10.319962 31413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:46:10.321098 31413 hybrid_clock.cc:648] HybridClock initialized: now 1714682770321086 us; error 25 us; skew 500 ppm
I20240502 20:46:10.323091 31413 webserver.cc:460] Webserver started at http://127.22.45.69:46521/ using document root <none> and password file <none>
I20240502 20:46:10.323647 31413 fs_manager.cc:359] Metadata directory not provided
I20240502 20:46:10.323716 31413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:46:10.327167 31413 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 20:46:10.329097 31427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:10.330153 31413 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:10.330276 31413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
format_stamp: "Formatted at 2024-05-02 20:45:55 on dist-test-slave-g7g5"
I20240502 20:46:10.330725 31413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:46:10.347049 31413 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:46:10.347656 31413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:46:10.347806 31413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:46:10.348331 31413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:46:10.349215 31413 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:46:10.349269 31413 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:10.349320 31413 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:46:10.349345 31413 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:10.358383 31413 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.69:43887
I20240502 20:46:10.358428 31540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.69:43887 every 8 connection(s)
I20240502 20:46:10.359364 31413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 20:46:10.363392 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 31413
I20240502 20:46:10.363766 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.70:46751
--local_ip_for_outbound_sockets=127.22.45.70
--tserver_master_addrs=127.22.45.126:34109
--webserver_port=35339
--webserver_interface=127.22.45.70
--builtin_ntp_servers=127.22.45.84:45597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:46:10.368394 31541 heartbeater.cc:347] Connected to a master server at 127.22.45.126:34109
I20240502 20:46:10.368607 31541 heartbeater.cc:464] Registering TS with master...
I20240502 20:46:10.369026 31541 heartbeater.cc:511] Master 127.22.45.126:34109 requested a full tablet report, sending...
I20240502 20:46:10.370029 29041 ts_manager.cc:194] Re-registered known tserver with Master: 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:10.370618 29041 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.69:50091
W20240502 20:46:10.472327 31544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:10.472539 31544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:46:10.472592 31544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:10.475564 31544 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:46:10.475631 31544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:46:10.475710 31544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.70
I20240502 20:46:10.479282 31544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:45597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.70:46751
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.22.45.70
--webserver_port=35339
--tserver_master_addrs=127.22.45.126:34109
--never_fsync=true
--heap_profile_path=/tmp/kudu.31544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.70
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:46:10.480073 31544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:46:10.481118 31544 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:46:10.488852 31552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:10.489092 31549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:10.489471 31550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:46:10.489943 31544 server_base.cc:1034] running on GCE node
I20240502 20:46:10.490275 31544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:46:10.490788 31544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:46:10.491927 31544 hybrid_clock.cc:648] HybridClock initialized: now 1714682770491909 us; error 30 us; skew 500 ppm
I20240502 20:46:10.493589 31544 webserver.cc:460] Webserver started at http://127.22.45.70:35339/ using document root <none> and password file <none>
I20240502 20:46:10.494131 31544 fs_manager.cc:359] Metadata directory not provided
I20240502 20:46:10.494201 31544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:46:10.497265 31544 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:10.498967 31558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:10.499866 31544 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:10.499967 31544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "315915efcf2b413ca7f9d874338e1b2d"
format_stamp: "Formatted at 2024-05-02 20:45:55 on dist-test-slave-g7g5"
I20240502 20:46:10.500370 31544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:46:10.511926 31544 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:46:10.512459 31544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:46:10.512583 31544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:46:10.513172 31544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:46:10.514163 31544 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:46:10.514214 31544 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:10.514258 31544 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:46:10.514288 31544 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:10.522528 31544 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.70:46751
I20240502 20:46:10.522562 31671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.70:46751 every 8 connection(s)
I20240502 20:46:10.523463 31544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 20:46:10.529688 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 31544
I20240502 20:46:10.530089 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.71:36787
--local_ip_for_outbound_sockets=127.22.45.71
--tserver_master_addrs=127.22.45.126:34109
--webserver_port=45465
--webserver_interface=127.22.45.71
--builtin_ntp_servers=127.22.45.84:45597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:46:10.532931 31672 heartbeater.cc:347] Connected to a master server at 127.22.45.126:34109
I20240502 20:46:10.533119 31672 heartbeater.cc:464] Registering TS with master...
I20240502 20:46:10.533558 31672 heartbeater.cc:511] Master 127.22.45.126:34109 requested a full tablet report, sending...
I20240502 20:46:10.534498 29041 ts_manager.cc:194] Re-registered known tserver with Master: 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:10.535266 29041 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.70:51785
W20240502 20:46:10.644479 31675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:10.644768 31675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:46:10.644855 31675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:10.648218 31675 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:46:10.648290 31675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:46:10.648361 31675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.71
I20240502 20:46:10.652211 31675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:45597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.71:36787
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.22.45.71
--webserver_port=45465
--tserver_master_addrs=127.22.45.126:34109
--never_fsync=true
--heap_profile_path=/tmp/kudu.31675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.71
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:46:10.653126 31675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:46:10.654301 31675 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:46:10.662654 31680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:10.663024 31681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:10.663388 31683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:46:10.663815 31675 server_base.cc:1034] running on GCE node
I20240502 20:46:10.664125 31675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:46:10.664638 31675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:46:10.665782 31675 hybrid_clock.cc:648] HybridClock initialized: now 1714682770665768 us; error 28 us; skew 500 ppm
I20240502 20:46:10.667593 31675 webserver.cc:460] Webserver started at http://127.22.45.71:45465/ using document root <none> and password file <none>
I20240502 20:46:10.668169 31675 fs_manager.cc:359] Metadata directory not provided
I20240502 20:46:10.668452 31675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:46:10.671726 31675 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20240502 20:46:10.673722 31689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:10.674765 31675 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:10.674861 31675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
format_stamp: "Formatted at 2024-05-02 20:45:55 on dist-test-slave-g7g5"
I20240502 20:46:10.675266 31675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:46:10.685678 31675 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:46:10.686378 31675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:46:10.686545 31675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:46:10.687064 31675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:46:10.687873 31675 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:46:10.687918 31675 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:10.687958 31675 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:46:10.687986 31675 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:10.696043 31675 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.71:36787
I20240502 20:46:10.696103 31802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.71:36787 every 8 connection(s)
I20240502 20:46:10.697026 31675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 20:46:10.705703 31803 heartbeater.cc:347] Connected to a master server at 127.22.45.126:34109
I20240502 20:46:10.705866 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 31675
I20240502 20:46:10.705936 31803 heartbeater.cc:464] Registering TS with master...
I20240502 20:46:10.706341 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.72:35207
--local_ip_for_outbound_sockets=127.22.45.72
--tserver_master_addrs=127.22.45.126:34109
--webserver_port=38267
--webserver_interface=127.22.45.72
--builtin_ntp_servers=127.22.45.84:45597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:46:10.706439 31803 heartbeater.cc:511] Master 127.22.45.126:34109 requested a full tablet report, sending...
I20240502 20:46:10.707491 29041 ts_manager.cc:194] Re-registered known tserver with Master: e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:10.708050 29041 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.71:56951
W20240502 20:46:10.813477 31806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:10.813689 31806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:46:10.813741 31806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:10.816924 31806 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:46:10.817001 31806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:46:10.817085 31806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.72
I20240502 20:46:10.820808 31806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:45597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.72:35207
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.22.45.72
--webserver_port=38267
--tserver_master_addrs=127.22.45.126:34109
--never_fsync=true
--heap_profile_path=/tmp/kudu.31806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.72
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:46:10.821841 31806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:46:10.823047 31806 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:46:10.831198 31814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:10.831437 31811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:10.831772 31812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:46:10.832226 31806 server_base.cc:1034] running on GCE node
I20240502 20:46:10.832629 31806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:46:10.833256 31806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:46:10.834415 31806 hybrid_clock.cc:648] HybridClock initialized: now 1714682770834324 us; error 102 us; skew 500 ppm
I20240502 20:46:10.836560 31806 webserver.cc:460] Webserver started at http://127.22.45.72:38267/ using document root <none> and password file <none>
I20240502 20:46:10.837231 31806 fs_manager.cc:359] Metadata directory not provided
I20240502 20:46:10.837318 31806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:46:10.841333 31806 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:10.843492 31820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:10.844727 31806 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 20:46:10.844831 31806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "e6b60803d0764d228c863e2da65e3691"
format_stamp: "Formatted at 2024-05-02 20:45:55 on dist-test-slave-g7g5"
I20240502 20:46:10.845304 31806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:46:10.861025 31806 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:46:10.861845 31806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:46:10.862032 31806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:46:10.862696 31806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:46:10.863739 31806 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:46:10.863801 31806 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:10.863850 31806 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:46:10.863879 31806 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:10.872907 31806 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.72:35207
I20240502 20:46:10.872936 31933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.72:35207 every 8 connection(s)
I20240502 20:46:10.873910 31806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 20:46:10.882582 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 31806
I20240502 20:46:10.883220 31934 heartbeater.cc:347] Connected to a master server at 127.22.45.126:34109
I20240502 20:46:10.883433 31934 heartbeater.cc:464] Registering TS with master...
I20240502 20:46:10.883942 31934 heartbeater.cc:511] Master 127.22.45.126:34109 requested a full tablet report, sending...
I20240502 20:46:10.884944 29041 ts_manager.cc:194] Re-registered known tserver with Master: e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:10.885695 29041 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.72:52471
I20240502 20:46:10.886222 29041 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:10.892787 29295 tablet_service.cc:1467] Processing CreateTablet for tablet e204d6e0ad034403a0c00a0ca97804af (DEFAULT_TABLE table=rebalancer_dynamic_table [id=3d698a79944046859c52952ef1699785]), partition=
I20240502 20:46:10.893074 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e204d6e0ad034403a0c00a0ca97804af. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:10.896275 31950 tablet_bootstrap.cc:492] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:10.899869 31950 tablet_bootstrap.cc:654] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:10.902009 31602 tablet_service.cc:1467] Processing CreateTablet for tablet e204d6e0ad034403a0c00a0ca97804af (DEFAULT_TABLE table=rebalancer_dynamic_table [id=3d698a79944046859c52952ef1699785]), partition=
I20240502 20:46:10.902791 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e204d6e0ad034403a0c00a0ca97804af. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:10.906903 31737 tablet_service.cc:1467] Processing CreateTablet for tablet e204d6e0ad034403a0c00a0ca97804af (DEFAULT_TABLE table=rebalancer_dynamic_table [id=3d698a79944046859c52952ef1699785]), partition=
I20240502 20:46:10.907611 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e204d6e0ad034403a0c00a0ca97804af. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:10.908869 31952 tablet_bootstrap.cc:492] T e204d6e0ad034403a0c00a0ca97804af P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:10.911168 31952 tablet_bootstrap.cc:654] T e204d6e0ad034403a0c00a0ca97804af P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:10.911885 31952 log.cc:824] T e204d6e0ad034403a0c00a0ca97804af P 315915efcf2b413ca7f9d874338e1b2d: Log is configured to *not* fsync() on all Append() calls
I20240502 20:46:10.912806 31950 tablet_bootstrap.cc:492] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:10.912910 31950 ts_tablet_manager.cc:1367] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 20:46:10.913336 31950 raft_consensus.cc:348] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:10.913443 31950 raft_consensus.cc:374] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:10.913475 31950 raft_consensus.cc:729] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:10.913595 31950 consensus_queue.cc:260] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:10.913873 31950 ts_tablet_manager.cc:1398] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:10.915086 31954 tablet_bootstrap.cc:492] T e204d6e0ad034403a0c00a0ca97804af P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:10.917660 31954 tablet_bootstrap.cc:654] T e204d6e0ad034403a0c00a0ca97804af P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:10.918440 31954 log.cc:824] T e204d6e0ad034403a0c00a0ca97804af P e7ab6b9053664ee5aa71c7978e26fe6a: Log is configured to *not* fsync() on all Append() calls
I20240502 20:46:10.919405 31952 tablet_bootstrap.cc:492] T e204d6e0ad034403a0c00a0ca97804af P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:10.919853 31952 ts_tablet_manager.cc:1367] T e204d6e0ad034403a0c00a0ca97804af P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20240502 20:46:10.920953 31954 tablet_bootstrap.cc:492] T e204d6e0ad034403a0c00a0ca97804af P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:10.921106 31954 ts_tablet_manager.cc:1367] T e204d6e0ad034403a0c00a0ca97804af P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.002s
I20240502 20:46:10.923807 31952 raft_consensus.cc:348] T e204d6e0ad034403a0c00a0ca97804af P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:10.924012 31952 raft_consensus.cc:374] T e204d6e0ad034403a0c00a0ca97804af P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:10.924064 31952 raft_consensus.cc:729] T e204d6e0ad034403a0c00a0ca97804af P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:10.924535 31952 consensus_queue.cc:260] T e204d6e0ad034403a0c00a0ca97804af P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:10.924890 31954 raft_consensus.cc:348] T e204d6e0ad034403a0c00a0ca97804af P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:10.925081 31954 raft_consensus.cc:374] T e204d6e0ad034403a0c00a0ca97804af P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:10.925133 31954 raft_consensus.cc:729] T e204d6e0ad034403a0c00a0ca97804af P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:10.925406 31672 heartbeater.cc:503] Master 127.22.45.126:34109 was elected leader, sending a full tablet report...
I20240502 20:46:10.925439 31952 ts_tablet_manager.cc:1398] T e204d6e0ad034403a0c00a0ca97804af P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240502 20:46:10.925563 31954 consensus_queue.cc:260] T e204d6e0ad034403a0c00a0ca97804af P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:10.926304 31954 ts_tablet_manager.cc:1398] T e204d6e0ad034403a0c00a0ca97804af P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20240502 20:46:10.926633 31803 heartbeater.cc:503] Master 127.22.45.126:34109 was elected leader, sending a full tablet report...
W20240502 20:46:10.947844 31804 tablet.cc:2369] T e204d6e0ad034403a0c00a0ca97804af P e7ab6b9053664ee5aa71c7978e26fe6a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 20:46:10.963413 31232 raft_consensus.cc:482] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:10.963536 31232 raft_consensus.cc:504] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:10.964012 31232 leader_election.cc:288] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:10.969730 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e204d6e0ad034403a0c00a0ca97804af" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:10.970064 31626 raft_consensus.cc:2454] T e204d6e0ad034403a0c00a0ca97804af P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:10.970541 29249 leader_election.cc:302] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 315915efcf2b413ca7f9d874338e1b2d; no voters:
I20240502 20:46:10.970854 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e204d6e0ad034403a0c00a0ca97804af" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:10.971114 31757 raft_consensus.cc:2454] T e204d6e0ad034403a0c00a0ca97804af P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:10.971149 31232 raft_consensus.cc:2781] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:10.971215 31232 raft_consensus.cc:482] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:10.971256 31232 raft_consensus.cc:3037] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:10.972322 31232 raft_consensus.cc:504] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:10.972702 31232 leader_election.cc:288] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:10.973234 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e204d6e0ad034403a0c00a0ca97804af" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:10.973358 31626 raft_consensus.cc:3037] T e204d6e0ad034403a0c00a0ca97804af P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:10.973524 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e204d6e0ad034403a0c00a0ca97804af" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:10.973629 31757 raft_consensus.cc:3037] T e204d6e0ad034403a0c00a0ca97804af P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:10.974422 31626 raft_consensus.cc:2454] T e204d6e0ad034403a0c00a0ca97804af P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:10.974489 31757 raft_consensus.cc:2454] T e204d6e0ad034403a0c00a0ca97804af P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:10.974754 29249 leader_election.cc:302] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 315915efcf2b413ca7f9d874338e1b2d; no voters:
I20240502 20:46:10.974941 31232 raft_consensus.cc:2781] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:10.975026 31232 raft_consensus.cc:686] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:10.975160 31232 consensus_queue.cc:237] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:10.977329 29041 catalog_manager.cc:5484] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
W20240502 20:46:11.024546 31673 tablet.cc:2369] T e204d6e0ad034403a0c00a0ca97804af P 315915efcf2b413ca7f9d874338e1b2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 20:46:11.093318 29041 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:11.098817 31626 raft_consensus.cc:1261] T e204d6e0ad034403a0c00a0ca97804af P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:11.099675 31757 raft_consensus.cc:1261] T e204d6e0ad034403a0c00a0ca97804af P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:11.100826 31328 consensus_queue.cc:1035] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:11.101141 31328 consensus_queue.cc:1035] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:11.106904 29038 catalog_manager.cc:5576] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751) does not have the latest schema for tablet e204d6e0ad034403a0c00a0ca97804af (table rebalancer_dynamic_table [id=3d698a79944046859c52952ef1699785]). Expected version 1 got 0
I20240502 20:46:11.107414 29035 catalog_manager.cc:5576] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787) does not have the latest schema for tablet e204d6e0ad034403a0c00a0ca97804af (table rebalancer_dynamic_table [id=3d698a79944046859c52952ef1699785]). Expected version 1 got 0
I20240502 20:46:11.108024 32002 mvcc.cc:205] Tried to move back new op lower bound from 7023340630414286848 to 7023340629915832320. Current Snapshot: MvccSnapshot[applied={T|T < 7023340630414286848}]
I20240502 20:46:11.108454 32007 tablet.cc:1722] T e204d6e0ad034403a0c00a0ca97804af P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:11.109212 29035 catalog_manager.cc:5576] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787) does not have the latest schema for tablet e204d6e0ad034403a0c00a0ca97804af (table rebalancer_dynamic_table [id=3d698a79944046859c52952ef1699785]). Expected version 1 got 0
I20240502 20:46:11.109323 32007 tablet.cc:1620] T e204d6e0ad034403a0c00a0ca97804af P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:11.109683 29038 catalog_manager.cc:5576] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751) does not have the latest schema for tablet e204d6e0ad034403a0c00a0ca97804af (table rebalancer_dynamic_table [id=3d698a79944046859c52952ef1699785]). Expected version 1 got 0
I20240502 20:46:11.111246 32004 mvcc.cc:205] Tried to move back new op lower bound from 7023340630414286848 to 7023340629915832320. Current Snapshot: MvccSnapshot[applied={T|T < 7023340630414286848}]
I20240502 20:46:11.107959 32006 tablet.cc:1722] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:11.113691 32006 tablet.cc:1620] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13: MemRowSet was empty: no flush needed.
I20240502 20:46:11.115191 32008 tablet.cc:1722] T e204d6e0ad034403a0c00a0ca97804af P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:11.115875 32008 tablet.cc:1620] T e204d6e0ad034403a0c00a0ca97804af P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:11.120540 29035 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=3d698a79944046859c52952ef1699785] alter complete (version 1)
I20240502 20:46:11.146014 29295 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 20:46:11.146020 29164 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 20:46:11.159987 31465 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:11.169605 31602 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:11.176445 31737 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:11.180629 29426 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 20:46:11.193603 29555 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:11.197790 31868 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:11.223127 29037 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:11.233817 32006 tablet.cc:1722] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:11.234158 32006 tablet.cc:1620] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13: MemRowSet was empty: no flush needed.
I20240502 20:46:11.234541 32008 tablet.cc:1722] T e204d6e0ad034403a0c00a0ca97804af P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:11.234817 32008 tablet.cc:1620] T e204d6e0ad034403a0c00a0ca97804af P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:11.235163 32007 tablet.cc:1722] T e204d6e0ad034403a0c00a0ca97804af P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:11.235461 32007 tablet.cc:1620] T e204d6e0ad034403a0c00a0ca97804af P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:11.239228 29037 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=3d698a79944046859c52952ef1699785] alter complete (version 2)
I20240502 20:46:11.343899 29042 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:11.351168 32006 tablet.cc:1722] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:11.352082 32008 tablet.cc:1722] T e204d6e0ad034403a0c00a0ca97804af P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:11.352317 32007 tablet.cc:1722] T e204d6e0ad034403a0c00a0ca97804af P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:11.357167 29042 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=3d698a79944046859c52952ef1699785] alter complete (version 3)
I20240502 20:46:11.368189 29295 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 20:46:11.370983 31602 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:11.373170 31541 heartbeater.cc:503] Master 127.22.45.126:34109 was elected leader, sending a full tablet report...
I20240502 20:46:11.375193 29164 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 20:46:11.383797 29034 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalance_test_table_0" } modify_external_catalogs: true
I20240502 20:46:11.383997 29034 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalance_test_table_0" } modify_external_catalogs: true
I20240502 20:46:11.385686 31737 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:11.385848 31465 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:11.387868 29036 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:11.387874 29034 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 5e6ae7bb0e174bd5bf8ca352bbf9d694
I20240502 20:46:11.390388 29034 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet a902eaf8bf274fb582550c1cf61d9465
I20240502 20:46:11.390630 29034 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 8912e9b8df214e2cb7688860793f1ab7
I20240502 20:46:11.391513 29557 tablet_service.cc:1514] Processing DeleteTablet for tablet 8912e9b8df214e2cb7688860793f1ab7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:11 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:11.391513 29555 tablet_service.cc:1514] Processing DeleteTablet for tablet a902eaf8bf274fb582550c1cf61d9465 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:11 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:11.393294 32088 tablet_replica.cc:330] T 8912e9b8df214e2cb7688860793f1ab7 P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:11.393626 32088 raft_consensus.cc:2229] T 8912e9b8df214e2cb7688860793f1ab7 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:11.393819 32088 raft_consensus.cc:2258] T 8912e9b8df214e2cb7688860793f1ab7 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:11.395500 29163 tablet_service.cc:1514] Processing DeleteTablet for tablet a902eaf8bf274fb582550c1cf61d9465 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:11 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:11.395500 29164 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e6ae7bb0e174bd5bf8ca352bbf9d694 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:11 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:11.395526 29162 tablet_service.cc:1514] Processing DeleteTablet for tablet 8912e9b8df214e2cb7688860793f1ab7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:11 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:11.397104 29426 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 20:46:11.400496 29555 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:11.400751 32088 ts_tablet_manager.cc:1875] T 8912e9b8df214e2cb7688860793f1ab7 P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:11.403596 32088 ts_tablet_manager.cc:1888] T 8912e9b8df214e2cb7688860793f1ab7 P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:11.403779 32088 log.cc:1196] T 8912e9b8df214e2cb7688860793f1ab7 P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/8912e9b8df214e2cb7688860793f1ab7
I20240502 20:46:11.404458 32088 ts_tablet_manager.cc:1909] T 8912e9b8df214e2cb7688860793f1ab7 P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:11.405026 32088 tablet_replica.cc:330] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:11.405263 32088 raft_consensus.cc:2229] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:11.405612 32088 raft_consensus.cc:2258] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:11.405984 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet 8912e9b8df214e2cb7688860793f1ab7 (table rebalance_test_table_0 [id=0ccbd3f0bec843cb859a6fc61e0ec2a9]) successfully deleted
I20240502 20:46:11.406306 32088 ts_tablet_manager.cc:1875] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:11.408309 32088 ts_tablet_manager.cc:1888] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:11.408450 32088 log.cc:1196] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/a902eaf8bf274fb582550c1cf61d9465
I20240502 20:46:11.408771 32088 ts_tablet_manager.cc:1909] T a902eaf8bf274fb582550c1cf61d9465 P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:11.410741 29164 tablet_service.cc:1467] Processing CreateTablet for tablet 6b675480162c4a9799db51ea4b6504a5 (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=bec3bd33882044f38d797093171eeb1d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:11.410827 29555 tablet_service.cc:1467] Processing CreateTablet for tablet 58d4026c208f45d8870a2eaa1c97efab (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=bec3bd33882044f38d797093171eeb1d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:11.411068 29555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58d4026c208f45d8870a2eaa1c97efab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:11.411057 29164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b675480162c4a9799db51ea4b6504a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:11.411623 29424 tablet_service.cc:1514] Processing DeleteTablet for tablet 8912e9b8df214e2cb7688860793f1ab7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:11 UTC) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:11.413512 29426 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e6ae7bb0e174bd5bf8ca352bbf9d694 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:11 UTC) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:11.413482 29424 tablet_service.cc:1467] Processing CreateTablet for tablet 6c29eeea8b9743e0a847a7cebc3c6e67 (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=bec3bd33882044f38d797093171eeb1d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:11.413771 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c29eeea8b9743e0a847a7cebc3c6e67. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:11.413832 29295 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e6ae7bb0e174bd5bf8ca352bbf9d694 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:11 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:11.414175 29294 tablet_service.cc:1514] Processing DeleteTablet for tablet a902eaf8bf274fb582550c1cf61d9465 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:11 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:11.410745 29163 tablet_service.cc:1467] Processing CreateTablet for tablet 58d4026c208f45d8870a2eaa1c97efab (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=bec3bd33882044f38d797093171eeb1d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:11.414598 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58d4026c208f45d8870a2eaa1c97efab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:11.416461 31602 tablet_service.cc:1467] Processing CreateTablet for tablet 58d4026c208f45d8870a2eaa1c97efab (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=bec3bd33882044f38d797093171eeb1d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:11.416797 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58d4026c208f45d8870a2eaa1c97efab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:11.417641 32095 tablet_replica.cc:330] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:11.417950 32095 raft_consensus.cc:2229] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:11.418282 32095 raft_consensus.cc:2258] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:11.419265 32095 ts_tablet_manager.cc:1875] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:11.420946 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet a902eaf8bf274fb582550c1cf61d9465 (table rebalance_test_table_0 [id=0ccbd3f0bec843cb859a6fc61e0ec2a9]) successfully deleted
I20240502 20:46:11.425212 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 6c29eeea8b9743e0a847a7cebc3c6e67 (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=bec3bd33882044f38d797093171eeb1d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:11.426294 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c29eeea8b9743e0a847a7cebc3c6e67. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:11.426638 32095 ts_tablet_manager.cc:1888] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:11.426798 32095 log.cc:1196] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/5e6ae7bb0e174bd5bf8ca352bbf9d694
I20240502 20:46:11.425220 31868 tablet_service.cc:1467] Processing CreateTablet for tablet 6b675480162c4a9799db51ea4b6504a5 (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=bec3bd33882044f38d797093171eeb1d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:11.427731 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b675480162c4a9799db51ea4b6504a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:11.428867 32095 ts_tablet_manager.cc:1909] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:11.430035 32095 tablet_replica.cc:330] T a902eaf8bf274fb582550c1cf61d9465 P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:11.430281 32095 raft_consensus.cc:2229] T a902eaf8bf274fb582550c1cf61d9465 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:11.430485 32095 raft_consensus.cc:2258] T a902eaf8bf274fb582550c1cf61d9465 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:11.431124 32095 ts_tablet_manager.cc:1875] T a902eaf8bf274fb582550c1cf61d9465 P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:11.433300 32095 ts_tablet_manager.cc:1888] T a902eaf8bf274fb582550c1cf61d9465 P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:11.433434 32095 log.cc:1196] T a902eaf8bf274fb582550c1cf61d9465 P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/a902eaf8bf274fb582550c1cf61d9465
I20240502 20:46:11.433800 32095 ts_tablet_manager.cc:1909] T a902eaf8bf274fb582550c1cf61d9465 P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:11.434723 31866 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:11.435345 32098 tablet_bootstrap.cc:492] T 6c29eeea8b9743e0a847a7cebc3c6e67 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:11.436465 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet 5e6ae7bb0e174bd5bf8ca352bbf9d694 (table rebalance_test_table_0 [id=0ccbd3f0bec843cb859a6fc61e0ec2a9]) successfully deleted
I20240502 20:46:11.436755 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet a902eaf8bf274fb582550c1cf61d9465 (table rebalance_test_table_0 [id=0ccbd3f0bec843cb859a6fc61e0ec2a9]) successfully deleted
I20240502 20:46:11.437637 32098 tablet_bootstrap.cc:654] T 6c29eeea8b9743e0a847a7cebc3c6e67 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:11.438475 32098 log.cc:824] T 6c29eeea8b9743e0a847a7cebc3c6e67 P e6b60803d0764d228c863e2da65e3691: Log is configured to *not* fsync() on all Append() calls
I20240502 20:46:11.439410 31465 tablet_service.cc:1467] Processing CreateTablet for tablet 6b675480162c4a9799db51ea4b6504a5 (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=bec3bd33882044f38d797093171eeb1d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:11.439410 31475 tablet_service.cc:1467] Processing CreateTablet for tablet 6c29eeea8b9743e0a847a7cebc3c6e67 (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=bec3bd33882044f38d797093171eeb1d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:11.440481 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c29eeea8b9743e0a847a7cebc3c6e67. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:11.442750 31465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b675480162c4a9799db51ea4b6504a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:11.445032 32097 tablet_bootstrap.cc:492] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:11.444794 32093 tablet_bootstrap.cc:492] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:11.446530 32097 tablet_bootstrap.cc:654] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:11.446607 32091 tablet_replica.cc:330] T 8912e9b8df214e2cb7688860793f1ab7 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:11.447085 32093 tablet_bootstrap.cc:654] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:11.446951 32094 tablet_bootstrap.cc:492] T 6b675480162c4a9799db51ea4b6504a5 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:11.447888 32096 tablet_bootstrap.cc:492] T 6c29eeea8b9743e0a847a7cebc3c6e67 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:11.448388 32091 raft_consensus.cc:2229] T 8912e9b8df214e2cb7688860793f1ab7 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:11.449458 32094 tablet_bootstrap.cc:654] T 6b675480162c4a9799db51ea4b6504a5 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:11.449821 32093 tablet_bootstrap.cc:492] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:11.449960 32093 ts_tablet_manager.cc:1367] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 20:46:11.450014 32091 raft_consensus.cc:2258] T 8912e9b8df214e2cb7688860793f1ab7 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:11.450949 32093 raft_consensus.cc:348] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:11.451157 32093 raft_consensus.cc:374] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:11.451238 32093 raft_consensus.cc:729] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:11.449786 32089 tablet_replica.cc:330] T a902eaf8bf274fb582550c1cf61d9465 P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:11.451501 32093 consensus_queue.cc:260] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:11.451586 32089 raft_consensus.cc:2229] T a902eaf8bf274fb582550c1cf61d9465 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:11.452579 32089 raft_consensus.cc:2258] T a902eaf8bf274fb582550c1cf61d9465 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:11.452896 32093 ts_tablet_manager.cc:1398] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:11.456254 32091 ts_tablet_manager.cc:1875] T 8912e9b8df214e2cb7688860793f1ab7 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:11.456640 32089 ts_tablet_manager.cc:1875] T a902eaf8bf274fb582550c1cf61d9465 P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:11.458853 32103 tablet_bootstrap.cc:492] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:11.460336 29036 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:11.460462 29036 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:11.461766 32103 tablet_bootstrap.cc:654] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:11.462682 32103 log.cc:824] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe: Log is configured to *not* fsync() on all Append() calls
I20240502 20:46:11.463132 32089 ts_tablet_manager.cc:1888] T a902eaf8bf274fb582550c1cf61d9465 P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:11.463821 29036 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet e204d6e0ad034403a0c00a0ca97804af
I20240502 20:46:11.464453 32103 tablet_bootstrap.cc:492] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:11.464489 31737 tablet_service.cc:1514] Processing DeleteTablet for tablet e204d6e0ad034403a0c00a0ca97804af with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:11 UTC) from {username='slave'} at 127.0.0.1:33864
I20240502 20:46:11.464617 32103 ts_tablet_manager.cc:1367] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.002s
I20240502 20:46:11.465401 32097 tablet_bootstrap.cc:492] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:11.465492 32097 ts_tablet_manager.cc:1367] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.001s
I20240502 20:46:11.466022 32097 raft_consensus.cc:348] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:11.466128 32097 raft_consensus.cc:374] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:11.466156 32097 raft_consensus.cc:729] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:11.466269 32097 consensus_queue.cc:260] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:11.448992 32096 tablet_bootstrap.cc:654] T 6c29eeea8b9743e0a847a7cebc3c6e67 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:11.466984 32097 ts_tablet_manager.cc:1398] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:11.467355 29294 tablet_service.cc:1514] Processing DeleteTablet for tablet e204d6e0ad034403a0c00a0ca97804af with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:11 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:11.467381 31602 tablet_service.cc:1514] Processing DeleteTablet for tablet e204d6e0ad034403a0c00a0ca97804af with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:11 UTC) from {username='slave'} at 127.0.0.1:58346
I20240502 20:46:11.467561 32095 tablet_replica.cc:330] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:11.467736 32095 raft_consensus.cc:2229] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:11.467958 32095 raft_consensus.cc:2258] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:11.469393 32091 ts_tablet_manager.cc:1888] T 8912e9b8df214e2cb7688860793f1ab7 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:11.469543 32091 log.cc:1196] T 8912e9b8df214e2cb7688860793f1ab7 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/8912e9b8df214e2cb7688860793f1ab7
I20240502 20:46:11.469862 32091 ts_tablet_manager.cc:1909] T 8912e9b8df214e2cb7688860793f1ab7 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting consensus metadata
I20240502 20:46:11.470960 32091 tablet_replica.cc:330] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:11.471012 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet 8912e9b8df214e2cb7688860793f1ab7 (table rebalance_test_table_0 [id=0ccbd3f0bec843cb859a6fc61e0ec2a9]) successfully deleted
I20240502 20:46:11.471177 32095 ts_tablet_manager.cc:1875] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:11.471449 32091 raft_consensus.cc:2229] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:11.471740 32103 raft_consensus.cc:348] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:11.471940 32103 raft_consensus.cc:374] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:11.472007 32103 raft_consensus.cc:729] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:11.472453 32103 consensus_queue.cc:260] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:11.473256 32103 ts_tablet_manager.cc:1398] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.009s user 0.002s sys 0.002s
I20240502 20:46:11.474232 32089 log.cc:1196] T a902eaf8bf274fb582550c1cf61d9465 P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/a902eaf8bf274fb582550c1cf61d9465
I20240502 20:46:11.474738 32103 tablet_bootstrap.cc:492] T 6c29eeea8b9743e0a847a7cebc3c6e67 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:11.475253 32095 ts_tablet_manager.cc:1888] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:11.475384 32089 ts_tablet_manager.cc:1909] T a902eaf8bf274fb582550c1cf61d9465 P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:11.476099 32095 log.cc:1196] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/e204d6e0ad034403a0c00a0ca97804af
I20240502 20:46:11.476487 32095 ts_tablet_manager.cc:1909] T e204d6e0ad034403a0c00a0ca97804af P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:11.476584 32103 tablet_bootstrap.cc:654] T 6c29eeea8b9743e0a847a7cebc3c6e67 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:11.477308 32105 tablet_replica.cc:330] T e204d6e0ad034403a0c00a0ca97804af P e7ab6b9053664ee5aa71c7978e26fe6a: stopping tablet replica
I20240502 20:46:11.477495 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet e204d6e0ad034403a0c00a0ca97804af (table rebalancer_dynamic_table_new_name [id=3d698a79944046859c52952ef1699785]) successfully deleted
I20240502 20:46:11.477690 32105 raft_consensus.cc:2229] T e204d6e0ad034403a0c00a0ca97804af P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:11.477897 32105 raft_consensus.cc:2258] T e204d6e0ad034403a0c00a0ca97804af P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:11.478590 32089 tablet_replica.cc:330] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:11.478646 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet a902eaf8bf274fb582550c1cf61d9465 (table rebalance_test_table_0 [id=0ccbd3f0bec843cb859a6fc61e0ec2a9]) successfully deleted
I20240502 20:46:11.478838 32089 raft_consensus.cc:2229] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:11.479110 32089 raft_consensus.cc:2258] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:11.479583 32094 tablet_bootstrap.cc:492] T 6b675480162c4a9799db51ea4b6504a5 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:11.479676 32094 ts_tablet_manager.cc:1367] T 6b675480162c4a9799db51ea4b6504a5 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.000s
I20240502 20:46:11.480074 32094 raft_consensus.cc:348] T 6b675480162c4a9799db51ea4b6504a5 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:11.480557 32094 raft_consensus.cc:374] T 6b675480162c4a9799db51ea4b6504a5 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:11.480621 32094 raft_consensus.cc:729] T 6b675480162c4a9799db51ea4b6504a5 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:11.480743 32094 consensus_queue.cc:260] T 6b675480162c4a9799db51ea4b6504a5 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:11.481112 32094 ts_tablet_manager.cc:1398] T 6b675480162c4a9799db51ea4b6504a5 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:11.481432 32094 tablet_bootstrap.cc:492] T 58d4026c208f45d8870a2eaa1c97efab P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:11.482393 32094 tablet_bootstrap.cc:654] T 58d4026c208f45d8870a2eaa1c97efab P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:11.484777 32106 tablet_replica.cc:330] T e204d6e0ad034403a0c00a0ca97804af P 315915efcf2b413ca7f9d874338e1b2d: stopping tablet replica
I20240502 20:46:11.485132 32106 raft_consensus.cc:2229] T e204d6e0ad034403a0c00a0ca97804af P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:11.485338 32106 raft_consensus.cc:2258] T e204d6e0ad034403a0c00a0ca97804af P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:11.486225 32105 ts_tablet_manager.cc:1875] T e204d6e0ad034403a0c00a0ca97804af P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:11.486224 32106 ts_tablet_manager.cc:1875] T e204d6e0ad034403a0c00a0ca97804af P 315915efcf2b413ca7f9d874338e1b2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:11.487049 32089 ts_tablet_manager.cc:1875] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:11.474129 32096 tablet_bootstrap.cc:492] T 6c29eeea8b9743e0a847a7cebc3c6e67 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:11.487303 32096 ts_tablet_manager.cc:1367] T 6c29eeea8b9743e0a847a7cebc3c6e67 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.039s user 0.002s sys 0.000s
I20240502 20:46:11.487802 32096 raft_consensus.cc:348] T 6c29eeea8b9743e0a847a7cebc3c6e67 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:11.487915 32096 raft_consensus.cc:374] T 6c29eeea8b9743e0a847a7cebc3c6e67 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:11.487943 32096 raft_consensus.cc:729] T 6c29eeea8b9743e0a847a7cebc3c6e67 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:11.488308 32096 consensus_queue.cc:260] T 6c29eeea8b9743e0a847a7cebc3c6e67 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:11.488692 32096 ts_tablet_manager.cc:1398] T 6c29eeea8b9743e0a847a7cebc3c6e67 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:11.488799 32106 ts_tablet_manager.cc:1888] T e204d6e0ad034403a0c00a0ca97804af P 315915efcf2b413ca7f9d874338e1b2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:11.488922 32106 log.cc:1196] T e204d6e0ad034403a0c00a0ca97804af P 315915efcf2b413ca7f9d874338e1b2d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal/wals/e204d6e0ad034403a0c00a0ca97804af
I20240502 20:46:11.489290 32106 ts_tablet_manager.cc:1909] T e204d6e0ad034403a0c00a0ca97804af P 315915efcf2b413ca7f9d874338e1b2d: Deleting consensus metadata
I20240502 20:46:11.489979 32089 ts_tablet_manager.cc:1888] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:11.490058 32089 log.cc:1196] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/8912e9b8df214e2cb7688860793f1ab7
I20240502 20:46:11.490311 32089 ts_tablet_manager.cc:1909] T 8912e9b8df214e2cb7688860793f1ab7 P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:11.490908 32089 tablet_replica.cc:330] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:11.491070 32089 raft_consensus.cc:2229] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:11.491222 32089 raft_consensus.cc:2258] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:11.487314 32091 raft_consensus.cc:2258] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:11.491680 32089 ts_tablet_manager.cc:1875] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:11.491906 32098 tablet_bootstrap.cc:492] T 6c29eeea8b9743e0a847a7cebc3c6e67 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:11.492117 32098 ts_tablet_manager.cc:1367] T 6c29eeea8b9743e0a847a7cebc3c6e67 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.057s user 0.004s sys 0.000s
I20240502 20:46:11.492143 32091 ts_tablet_manager.cc:1875] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:11.493683 32091 ts_tablet_manager.cc:1888] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:11.493681 32089 ts_tablet_manager.cc:1888] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:11.493754 32089 log.cc:1196] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/5e6ae7bb0e174bd5bf8ca352bbf9d694
I20240502 20:46:11.493754 32091 log.cc:1196] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/5e6ae7bb0e174bd5bf8ca352bbf9d694
I20240502 20:46:11.493961 32091 ts_tablet_manager.cc:1909] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting consensus metadata
I20240502 20:46:11.494006 32089 ts_tablet_manager.cc:1909] T 5e6ae7bb0e174bd5bf8ca352bbf9d694 P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:11.494581 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet 5e6ae7bb0e174bd5bf8ca352bbf9d694 (table rebalance_test_table_0 [id=0ccbd3f0bec843cb859a6fc61e0ec2a9]) successfully deleted
I20240502 20:46:11.494647 29027 catalog_manager.cc:4830] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): tablet e204d6e0ad034403a0c00a0ca97804af (table rebalancer_dynamic_table_new_name [id=3d698a79944046859c52952ef1699785]) successfully deleted
I20240502 20:46:11.494954 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 8912e9b8df214e2cb7688860793f1ab7 (table rebalance_test_table_0 [id=0ccbd3f0bec843cb859a6fc61e0ec2a9]) successfully deleted
I20240502 20:46:11.495088 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 5e6ae7bb0e174bd5bf8ca352bbf9d694 (table rebalance_test_table_0 [id=0ccbd3f0bec843cb859a6fc61e0ec2a9]) successfully deleted
I20240502 20:46:11.496562 32105 ts_tablet_manager.cc:1888] T e204d6e0ad034403a0c00a0ca97804af P e7ab6b9053664ee5aa71c7978e26fe6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:11.496659 32105 log.cc:1196] T e204d6e0ad034403a0c00a0ca97804af P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/wals/e204d6e0ad034403a0c00a0ca97804af
I20240502 20:46:11.496981 32105 ts_tablet_manager.cc:1909] T e204d6e0ad034403a0c00a0ca97804af P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting consensus metadata
I20240502 20:46:11.497414 32098 raft_consensus.cc:348] T 6c29eeea8b9743e0a847a7cebc3c6e67 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:11.497624 32098 raft_consensus.cc:374] T 6c29eeea8b9743e0a847a7cebc3c6e67 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:11.497694 32098 raft_consensus.cc:729] T 6c29eeea8b9743e0a847a7cebc3c6e67 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:11.498034 29026 catalog_manager.cc:4830] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): tablet e204d6e0ad034403a0c00a0ca97804af (table rebalancer_dynamic_table_new_name [id=3d698a79944046859c52952ef1699785]) successfully deleted
I20240502 20:46:11.498133 32098 consensus_queue.cc:260] T 6c29eeea8b9743e0a847a7cebc3c6e67 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:11.498751 32103 tablet_bootstrap.cc:492] T 6c29eeea8b9743e0a847a7cebc3c6e67 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:11.498806 32094 tablet_bootstrap.cc:492] T 58d4026c208f45d8870a2eaa1c97efab P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:11.498945 32094 ts_tablet_manager.cc:1367] T 58d4026c208f45d8870a2eaa1c97efab P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240502 20:46:11.499006 31934 heartbeater.cc:503] Master 127.22.45.126:34109 was elected leader, sending a full tablet report...
I20240502 20:46:11.499526 32094 raft_consensus.cc:348] T 58d4026c208f45d8870a2eaa1c97efab P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:11.499732 32094 raft_consensus.cc:374] T 58d4026c208f45d8870a2eaa1c97efab P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:11.499821 32094 raft_consensus.cc:729] T 58d4026c208f45d8870a2eaa1c97efab P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:11.498867 32103 ts_tablet_manager.cc:1367] T 6c29eeea8b9743e0a847a7cebc3c6e67 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.001s
I20240502 20:46:11.500311 32094 consensus_queue.cc:260] T 58d4026c208f45d8870a2eaa1c97efab P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:11.500754 32094 ts_tablet_manager.cc:1398] T 58d4026c208f45d8870a2eaa1c97efab P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:11.500720 32103 raft_consensus.cc:348] T 6c29eeea8b9743e0a847a7cebc3c6e67 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:11.500829 32103 raft_consensus.cc:374] T 6c29eeea8b9743e0a847a7cebc3c6e67 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:11.501026 32103 raft_consensus.cc:729] T 6c29eeea8b9743e0a847a7cebc3c6e67 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:11.501185 32103 consensus_queue.cc:260] T 6c29eeea8b9743e0a847a7cebc3c6e67 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:11.501578 32103 ts_tablet_manager.cc:1398] T 6c29eeea8b9743e0a847a7cebc3c6e67 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:11.501976 32098 ts_tablet_manager.cc:1398] T 6c29eeea8b9743e0a847a7cebc3c6e67 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20240502 20:46:11.510593 32098 tablet_bootstrap.cc:492] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:11.511616 32098 tablet_bootstrap.cc:654] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:11.513064 32098 tablet_bootstrap.cc:492] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:11.513157 32098 ts_tablet_manager.cc:1367] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:11.513701 32098 raft_consensus.cc:348] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:11.513803 32098 raft_consensus.cc:374] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:11.513823 32098 raft_consensus.cc:729] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:11.513936 32098 consensus_queue.cc:260] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:11.514189 32098 ts_tablet_manager.cc:1398] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:11.565968 29036 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:11.573778 31737 tablet_service.cc:1467] Processing CreateTablet for tablet c25af37f2da84a3d85f92d1aa1a1e092 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=ed288929859c426894df1426364b75f9]), partition=
I20240502 20:46:11.573778 29424 tablet_service.cc:1467] Processing CreateTablet for tablet c25af37f2da84a3d85f92d1aa1a1e092 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=ed288929859c426894df1426364b75f9]), partition=
I20240502 20:46:11.574049 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c25af37f2da84a3d85f92d1aa1a1e092. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:11.574050 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c25af37f2da84a3d85f92d1aa1a1e092. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:11.576185 32096 tablet_bootstrap.cc:492] T c25af37f2da84a3d85f92d1aa1a1e092 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:11.576272 32154 tablet_bootstrap.cc:492] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:11.577067 32096 tablet_bootstrap.cc:654] T c25af37f2da84a3d85f92d1aa1a1e092 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:11.577067 32154 tablet_bootstrap.cc:654] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:11.577075 31602 tablet_service.cc:1467] Processing CreateTablet for tablet c25af37f2da84a3d85f92d1aa1a1e092 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=ed288929859c426894df1426364b75f9]), partition=
I20240502 20:46:11.577327 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c25af37f2da84a3d85f92d1aa1a1e092. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:11.578799 32096 tablet_bootstrap.cc:492] T c25af37f2da84a3d85f92d1aa1a1e092 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:11.578881 32096 ts_tablet_manager.cc:1367] T c25af37f2da84a3d85f92d1aa1a1e092 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:11.579201 32097 tablet_bootstrap.cc:492] T c25af37f2da84a3d85f92d1aa1a1e092 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:11.579286 32096 raft_consensus.cc:348] T c25af37f2da84a3d85f92d1aa1a1e092 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:11.579424 32096 raft_consensus.cc:374] T c25af37f2da84a3d85f92d1aa1a1e092 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:11.579448 32096 raft_consensus.cc:729] T c25af37f2da84a3d85f92d1aa1a1e092 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:11.579551 32096 consensus_queue.cc:260] T c25af37f2da84a3d85f92d1aa1a1e092 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:11.579807 32096 ts_tablet_manager.cc:1398] T c25af37f2da84a3d85f92d1aa1a1e092 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:11.579991 32097 tablet_bootstrap.cc:654] T c25af37f2da84a3d85f92d1aa1a1e092 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:11.583782 32097 tablet_bootstrap.cc:492] T c25af37f2da84a3d85f92d1aa1a1e092 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:11.583880 32097 ts_tablet_manager.cc:1367] T c25af37f2da84a3d85f92d1aa1a1e092 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240502 20:46:11.584334 32097 raft_consensus.cc:348] T c25af37f2da84a3d85f92d1aa1a1e092 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:11.584439 32097 raft_consensus.cc:374] T c25af37f2da84a3d85f92d1aa1a1e092 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:11.584465 32097 raft_consensus.cc:729] T c25af37f2da84a3d85f92d1aa1a1e092 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:11.584580 32097 consensus_queue.cc:260] T c25af37f2da84a3d85f92d1aa1a1e092 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:11.584939 32097 ts_tablet_manager.cc:1398] T c25af37f2da84a3d85f92d1aa1a1e092 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:11.586906 32154 tablet_bootstrap.cc:492] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:11.587018 32154 ts_tablet_manager.cc:1367] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240502 20:46:11.587466 32154 raft_consensus.cc:348] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:11.587555 32154 raft_consensus.cc:374] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:11.587584 32154 raft_consensus.cc:729] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:11.587723 32154 consensus_queue.cc:260] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:11.587999 32154 ts_tablet_manager.cc:1398] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20240502 20:46:11.611289 31542 tablet.cc:2369] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 20:46:11.625164 31935 tablet.cc:2369] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 20:46:11.634485 31956 raft_consensus.cc:482] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:11.634631 31956 raft_consensus.cc:504] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:11.635852 31956 leader_election.cc:288] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:11.639684 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58d4026c208f45d8870a2eaa1c97efab" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:11.639698 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58d4026c208f45d8870a2eaa1c97efab" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:11.639851 29182 raft_consensus.cc:2454] T 58d4026c208f45d8870a2eaa1c97efab P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:11.639866 29574 raft_consensus.cc:2454] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:11.640203 31562 leader_election.cc:302] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:11.640441 31956 raft_consensus.cc:2781] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:11.640516 31956 raft_consensus.cc:482] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:11.640558 31956 raft_consensus.cc:3037] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:11.641360 31956 raft_consensus.cc:504] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:11.641685 31956 leader_election.cc:288] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:11.641968 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58d4026c208f45d8870a2eaa1c97efab" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:11.642009 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58d4026c208f45d8870a2eaa1c97efab" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:11.642082 29574 raft_consensus.cc:3037] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:11.642133 29182 raft_consensus.cc:3037] T 58d4026c208f45d8870a2eaa1c97efab P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:11.642906 29182 raft_consensus.cc:2454] T 58d4026c208f45d8870a2eaa1c97efab P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:11.642946 29574 raft_consensus.cc:2454] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:11.643167 31562 leader_election.cc:302] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:11.643337 31956 raft_consensus.cc:2781] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:11.643536 31956 raft_consensus.cc:686] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Becoming Leader. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Running, Role: LEADER
I20240502 20:46:11.643760 31956 consensus_queue.cc:237] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:11.645625 29033 catalog_manager.cc:5484] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70). New cstate: current_term: 1 leader_uuid: "315915efcf2b413ca7f9d874338e1b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:11.679942 32108 raft_consensus.cc:482] T 6c29eeea8b9743e0a847a7cebc3c6e67 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:11.680155 32108 raft_consensus.cc:504] T 6c29eeea8b9743e0a847a7cebc3c6e67 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:11.681443 32108 leader_election.cc:288] T 6c29eeea8b9743e0a847a7cebc3c6e67 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:11.685343 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c29eeea8b9743e0a847a7cebc3c6e67" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:11.685514 29446 raft_consensus.cc:2454] T 6c29eeea8b9743e0a847a7cebc3c6e67 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:11.686090 31430 leader_election.cc:302] T 6c29eeea8b9743e0a847a7cebc3c6e67 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:11.686283 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c29eeea8b9743e0a847a7cebc3c6e67" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:11.686379 32108 raft_consensus.cc:2781] T 6c29eeea8b9743e0a847a7cebc3c6e67 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:11.686489 32108 raft_consensus.cc:482] T 6c29eeea8b9743e0a847a7cebc3c6e67 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:11.686543 32108 raft_consensus.cc:3037] T 6c29eeea8b9743e0a847a7cebc3c6e67 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:11.686630 31888 raft_consensus.cc:2454] T 6c29eeea8b9743e0a847a7cebc3c6e67 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:11.687523 32108 raft_consensus.cc:504] T 6c29eeea8b9743e0a847a7cebc3c6e67 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:11.687845 32108 leader_election.cc:288] T 6c29eeea8b9743e0a847a7cebc3c6e67 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Requested vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:11.688145 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c29eeea8b9743e0a847a7cebc3c6e67" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:11.688187 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c29eeea8b9743e0a847a7cebc3c6e67" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:11.688247 29446 raft_consensus.cc:3037] T 6c29eeea8b9743e0a847a7cebc3c6e67 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:11.688290 31888 raft_consensus.cc:3037] T 6c29eeea8b9743e0a847a7cebc3c6e67 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:11.689054 29446 raft_consensus.cc:2454] T 6c29eeea8b9743e0a847a7cebc3c6e67 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:11.689191 31888 raft_consensus.cc:2454] T 6c29eeea8b9743e0a847a7cebc3c6e67 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:11.689307 31430 leader_election.cc:302] T 6c29eeea8b9743e0a847a7cebc3c6e67 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:11.689463 32108 raft_consensus.cc:2781] T 6c29eeea8b9743e0a847a7cebc3c6e67 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:11.689679 32108 raft_consensus.cc:686] T 6c29eeea8b9743e0a847a7cebc3c6e67 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Becoming Leader. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Running, Role: LEADER
I20240502 20:46:11.690017 32108 consensus_queue.cc:237] T 6c29eeea8b9743e0a847a7cebc3c6e67 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:11.692592 29033 catalog_manager.cc:5484] T 6c29eeea8b9743e0a847a7cebc3c6e67 P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69). New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:11.815429 31957 raft_consensus.cc:482] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:11.815560 31957 raft_consensus.cc:504] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:11.816655 31957 leader_election.cc:288] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:11.820565 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c25af37f2da84a3d85f92d1aa1a1e092" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:11.820741 31626 raft_consensus.cc:2454] T c25af37f2da84a3d85f92d1aa1a1e092 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:11.820703 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c25af37f2da84a3d85f92d1aa1a1e092" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:11.820847 29446 raft_consensus.cc:2454] T c25af37f2da84a3d85f92d1aa1a1e092 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:11.821125 31691 leader_election.cc:302] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:11.821367 31957 raft_consensus.cc:2781] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:11.821424 31957 raft_consensus.cc:482] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:11.821460 31957 raft_consensus.cc:3037] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:11.822408 31957 raft_consensus.cc:504] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:11.822829 31957 leader_election.cc:288] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Requested vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:11.823115 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c25af37f2da84a3d85f92d1aa1a1e092" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:11.823174 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c25af37f2da84a3d85f92d1aa1a1e092" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:11.823232 29446 raft_consensus.cc:3037] T c25af37f2da84a3d85f92d1aa1a1e092 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:11.823292 31626 raft_consensus.cc:3037] T c25af37f2da84a3d85f92d1aa1a1e092 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:11.824020 29446 raft_consensus.cc:2454] T c25af37f2da84a3d85f92d1aa1a1e092 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:11.824057 31626 raft_consensus.cc:2454] T c25af37f2da84a3d85f92d1aa1a1e092 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:11.824301 31692 leader_election.cc:302] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dbcdc6ac635b44b185c3eba63f66e9d6, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:11.824502 31957 raft_consensus.cc:2781] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:11.824684 31957 raft_consensus.cc:686] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Becoming Leader. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Running, Role: LEADER
I20240502 20:46:11.824930 31957 consensus_queue.cc:237] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:11.826963 29033 catalog_manager.cc:5484] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: term changed from 0 to 1, leader changed from <none> to e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71). New cstate: current_term: 1 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:11.875416 32111 raft_consensus.cc:482] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:11.875538 32111 raft_consensus.cc:504] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:11.876673 32111 leader_election.cc:288] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:11.880724 32108 raft_consensus.cc:482] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:11.880833 32108 raft_consensus.cc:504] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:11.881315 32108 leader_election.cc:288] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:11.881302 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b675480162c4a9799db51ea4b6504a5" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:11.881433 29175 raft_consensus.cc:2454] T 6b675480162c4a9799db51ea4b6504a5 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:11.881636 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b675480162c4a9799db51ea4b6504a5" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:11.881759 31888 raft_consensus.cc:2454] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:11.882155 31824 leader_election.cc:302] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:11.882412 32111 raft_consensus.cc:2781] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:11.882494 32111 raft_consensus.cc:482] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:11.882534 32111 raft_consensus.cc:3037] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:11.882579 31429 leader_election.cc:302] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:11.882763 32108 raft_consensus.cc:2781] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:11.882810 32108 raft_consensus.cc:482] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:11.882840 32108 raft_consensus.cc:3037] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:11.882838 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b675480162c4a9799db51ea4b6504a5" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:11.883524 32111 raft_consensus.cc:504] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:11.883808 32108 raft_consensus.cc:504] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:11.883879 32111 leader_election.cc:288] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:11.884150 32108 leader_election.cc:288] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:11.884249 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b675480162c4a9799db51ea4b6504a5" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:11.884337 31495 raft_consensus.cc:2379] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e6b60803d0764d228c863e2da65e3691 in current term 1: Already voted for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in this term.
I20240502 20:46:11.884516 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b675480162c4a9799db51ea4b6504a5" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:11.884625 29175 raft_consensus.cc:3037] T 6b675480162c4a9799db51ea4b6504a5 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:11.884989 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b675480162c4a9799db51ea4b6504a5" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:11.885133 31888 raft_consensus.cc:2379] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2a9c1af8cf814030bde56b67ca5c2fbe in current term 1: Already voted for candidate e6b60803d0764d228c863e2da65e3691 in this term.
I20240502 20:46:11.885589 29175 raft_consensus.cc:2454] T 6b675480162c4a9799db51ea4b6504a5 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:11.885875 31824 leader_election.cc:302] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e6b60803d0764d228c863e2da65e3691; no voters: 2a9c1af8cf814030bde56b67ca5c2fbe
I20240502 20:46:11.886590 32111 raft_consensus.cc:2781] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:11.886788 32111 raft_consensus.cc:686] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Becoming Leader. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Running, Role: LEADER
I20240502 20:46:11.887095 32111 consensus_queue.cc:237] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:11.887229 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b675480162c4a9799db51ea4b6504a5" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:11.887341 29182 raft_consensus.cc:2379] T 6b675480162c4a9799db51ea4b6504a5 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2a9c1af8cf814030bde56b67ca5c2fbe in current term 1: Already voted for candidate e6b60803d0764d228c863e2da65e3691 in this term.
I20240502 20:46:11.887660 31431 leader_election.cc:302] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe; no voters: 7796ea3fba294d17a34386873969431d, e6b60803d0764d228c863e2da65e3691
I20240502 20:46:11.887849 32108 raft_consensus.cc:2726] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240502 20:46:11.887928 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b675480162c4a9799db51ea4b6504a5" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:11.888036 29175 raft_consensus.cc:2379] T 6b675480162c4a9799db51ea4b6504a5 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2a9c1af8cf814030bde56b67ca5c2fbe in current term 1: Already voted for candidate e6b60803d0764d228c863e2da65e3691 in this term.
I20240502 20:46:11.890883 29033 catalog_manager.cc:5484] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6b60803d0764d228c863e2da65e3691 (127.22.45.72). New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:11.892653 29039 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalance_test_table_2" } modify_external_catalogs: true
I20240502 20:46:11.892760 29039 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalance_test_table_2" } modify_external_catalogs: true
I20240502 20:46:11.894834 29039 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 1227d2743d824309949815354bcf7fce
I20240502 20:46:11.895134 29039 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 68eafb3578684972acc7ebdfe309a955
I20240502 20:46:11.895375 29039 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet e1459e78fecb466db1aa62cb7f53326a
I20240502 20:46:11.895639 29424 tablet_service.cc:1514] Processing DeleteTablet for tablet 68eafb3578684972acc7ebdfe309a955 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:11 UTC) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:11.895721 29555 tablet_service.cc:1514] Processing DeleteTablet for tablet 1227d2743d824309949815354bcf7fce with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:11 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:11.895825 32091 tablet_replica.cc:330] T 68eafb3578684972acc7ebdfe309a955 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:11.895902 32088 tablet_replica.cc:330] T 1227d2743d824309949815354bcf7fce P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:11.895964 32091 raft_consensus.cc:2229] T 68eafb3578684972acc7ebdfe309a955 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:11.896041 32088 raft_consensus.cc:2229] T 1227d2743d824309949815354bcf7fce P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:11.896097 32091 raft_consensus.cc:2258] T 68eafb3578684972acc7ebdfe309a955 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:11.896148 32088 raft_consensus.cc:2258] T 1227d2743d824309949815354bcf7fce P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:11.896626 32088 ts_tablet_manager.cc:1875] T 1227d2743d824309949815354bcf7fce P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:11.896632 32091 ts_tablet_manager.cc:1875] T 68eafb3578684972acc7ebdfe309a955 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:11.898286 32091 ts_tablet_manager.cc:1888] T 68eafb3578684972acc7ebdfe309a955 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:11.898340 32088 ts_tablet_manager.cc:1888] T 1227d2743d824309949815354bcf7fce P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:11.898360 32091 log.cc:1196] T 68eafb3578684972acc7ebdfe309a955 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/68eafb3578684972acc7ebdfe309a955
I20240502 20:46:11.898406 32088 log.cc:1196] T 1227d2743d824309949815354bcf7fce P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/1227d2743d824309949815354bcf7fce
I20240502 20:46:11.898588 32091 ts_tablet_manager.cc:1909] T 68eafb3578684972acc7ebdfe309a955 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting consensus metadata
I20240502 20:46:11.898638 32088 ts_tablet_manager.cc:1909] T 1227d2743d824309949815354bcf7fce P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:11.899060 29555 tablet_service.cc:1514] Processing DeleteTablet for tablet 68eafb3578684972acc7ebdfe309a955 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:11 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:11.899135 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet 68eafb3578684972acc7ebdfe309a955 (table rebalance_test_table_2 [id=0c689bc5ef924b3eae28a1e2f63671df]) successfully deleted
I20240502 20:46:11.899195 29557 tablet_service.cc:1514] Processing DeleteTablet for tablet e1459e78fecb466db1aa62cb7f53326a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:11 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:11.899308 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet 1227d2743d824309949815354bcf7fce (table rebalance_test_table_2 [id=0c689bc5ef924b3eae28a1e2f63671df]) successfully deleted
I20240502 20:46:11.899508 29294 tablet_service.cc:1514] Processing DeleteTablet for tablet 1227d2743d824309949815354bcf7fce with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:11 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:11.899681 32095 tablet_replica.cc:330] T 1227d2743d824309949815354bcf7fce P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:11.899830 32095 raft_consensus.cc:2229] T 1227d2743d824309949815354bcf7fce P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:11.899813 29163 tablet_service.cc:1514] Processing DeleteTablet for tablet 1227d2743d824309949815354bcf7fce with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:11 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:11.899958 32095 raft_consensus.cc:2258] T 1227d2743d824309949815354bcf7fce P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:11.899964 29164 tablet_service.cc:1514] Processing DeleteTablet for tablet 68eafb3578684972acc7ebdfe309a955 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:11 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:11.900067 32089 tablet_replica.cc:330] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:11.900194 32089 raft_consensus.cc:2229] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:11.900377 32089 raft_consensus.cc:2258] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:11.900456 32095 ts_tablet_manager.cc:1875] T 1227d2743d824309949815354bcf7fce P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:11.900884 32089 ts_tablet_manager.cc:1875] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:11.902498 32095 ts_tablet_manager.cc:1888] T 1227d2743d824309949815354bcf7fce P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:11.902561 32095 log.cc:1196] T 1227d2743d824309949815354bcf7fce P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/1227d2743d824309949815354bcf7fce
I20240502 20:46:11.902806 32095 ts_tablet_manager.cc:1909] T 1227d2743d824309949815354bcf7fce P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:11.902948 32089 ts_tablet_manager.cc:1888] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:11.903003 32089 log.cc:1196] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/1227d2743d824309949815354bcf7fce
I20240502 20:46:11.903241 32089 ts_tablet_manager.cc:1909] T 1227d2743d824309949815354bcf7fce P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:11.899523 29295 tablet_service.cc:1514] Processing DeleteTablet for tablet e1459e78fecb466db1aa62cb7f53326a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:11 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:11.903409 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet 1227d2743d824309949815354bcf7fce (table rebalance_test_table_2 [id=0c689bc5ef924b3eae28a1e2f63671df]) successfully deleted
I20240502 20:46:11.903499 32095 tablet_replica.cc:330] T e1459e78fecb466db1aa62cb7f53326a P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:11.903611 32095 raft_consensus.cc:2229] T e1459e78fecb466db1aa62cb7f53326a P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:11.903728 32089 tablet_replica.cc:330] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:11.903755 32095 raft_consensus.cc:2258] T e1459e78fecb466db1aa62cb7f53326a P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:11.903851 32089 raft_consensus.cc:2229] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:11.904011 32089 raft_consensus.cc:2258] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:11.904186 32095 ts_tablet_manager.cc:1875] T e1459e78fecb466db1aa62cb7f53326a P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:11.904407 32089 ts_tablet_manager.cc:1875] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:11.905542 32095 ts_tablet_manager.cc:1888] T e1459e78fecb466db1aa62cb7f53326a P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:11.905606 32095 log.cc:1196] T e1459e78fecb466db1aa62cb7f53326a P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/e1459e78fecb466db1aa62cb7f53326a
I20240502 20:46:11.905757 32089 ts_tablet_manager.cc:1888] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:11.905786 32095 ts_tablet_manager.cc:1909] T e1459e78fecb466db1aa62cb7f53326a P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:11.905822 32089 log.cc:1196] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/68eafb3578684972acc7ebdfe309a955
I20240502 20:46:11.906020 32089 ts_tablet_manager.cc:1909] T 68eafb3578684972acc7ebdfe309a955 P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:11.906294 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet e1459e78fecb466db1aa62cb7f53326a (table rebalance_test_table_2 [id=0c689bc5ef924b3eae28a1e2f63671df]) successfully deleted
I20240502 20:46:11.906682 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 1227d2743d824309949815354bcf7fce (table rebalance_test_table_2 [id=0c689bc5ef924b3eae28a1e2f63671df]) successfully deleted
I20240502 20:46:11.899209 32088 tablet_replica.cc:330] T 68eafb3578684972acc7ebdfe309a955 P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:11.906831 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 68eafb3578684972acc7ebdfe309a955 (table rebalance_test_table_2 [id=0c689bc5ef924b3eae28a1e2f63671df]) successfully deleted
I20240502 20:46:11.906833 29164 tablet_service.cc:1514] Processing DeleteTablet for tablet e1459e78fecb466db1aa62cb7f53326a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:11 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:11.906896 32088 raft_consensus.cc:2229] T 68eafb3578684972acc7ebdfe309a955 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:11.906984 32089 tablet_replica.cc:330] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:11.907020 32088 raft_consensus.cc:2258] T 68eafb3578684972acc7ebdfe309a955 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:11.907126 32089 raft_consensus.cc:2229] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:11.907294 32089 raft_consensus.cc:2258] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:11.907455 32088 ts_tablet_manager.cc:1875] T 68eafb3578684972acc7ebdfe309a955 P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:11.907713 32089 ts_tablet_manager.cc:1875] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:11.909404 32089 ts_tablet_manager.cc:1888] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:11.909406 32088 ts_tablet_manager.cc:1888] T 68eafb3578684972acc7ebdfe309a955 P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:11.909473 32088 log.cc:1196] T 68eafb3578684972acc7ebdfe309a955 P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/68eafb3578684972acc7ebdfe309a955
I20240502 20:46:11.909513 32089 log.cc:1196] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/e1459e78fecb466db1aa62cb7f53326a
I20240502 20:46:11.909739 32088 ts_tablet_manager.cc:1909] T 68eafb3578684972acc7ebdfe309a955 P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:11.909754 32089 ts_tablet_manager.cc:1909] T e1459e78fecb466db1aa62cb7f53326a P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:11.910204 32088 tablet_replica.cc:330] T e1459e78fecb466db1aa62cb7f53326a P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:11.910337 32088 raft_consensus.cc:2229] T e1459e78fecb466db1aa62cb7f53326a P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:11.910347 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet e1459e78fecb466db1aa62cb7f53326a (table rebalance_test_table_2 [id=0c689bc5ef924b3eae28a1e2f63671df]) successfully deleted
I20240502 20:46:11.910522 32088 raft_consensus.cc:2258] T e1459e78fecb466db1aa62cb7f53326a P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:11.910602 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet 68eafb3578684972acc7ebdfe309a955 (table rebalance_test_table_2 [id=0c689bc5ef924b3eae28a1e2f63671df]) successfully deleted
I20240502 20:46:11.910917 32088 ts_tablet_manager.cc:1875] T e1459e78fecb466db1aa62cb7f53326a P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:11.915777 32088 ts_tablet_manager.cc:1888] T e1459e78fecb466db1aa62cb7f53326a P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:11.915876 32088 log.cc:1196] T e1459e78fecb466db1aa62cb7f53326a P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/e1459e78fecb466db1aa62cb7f53326a
I20240502 20:46:11.916122 32088 ts_tablet_manager.cc:1909] T e1459e78fecb466db1aa62cb7f53326a P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:11.917706 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet e1459e78fecb466db1aa62cb7f53326a (table rebalance_test_table_2 [id=0c689bc5ef924b3eae28a1e2f63671df]) successfully deleted
I20240502 20:46:11.920428 31602 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:46:11.922677 31475 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:46:11.927634 29164 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:11.927933 31867 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:46:11.935709 29039 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:11.937034 29295 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 20:46:11.941902 29446 raft_consensus.cc:1261] T c25af37f2da84a3d85f92d1aa1a1e092 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer e7ab6b9053664ee5aa71c7978e26fe6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:11.941943 31626 raft_consensus.cc:1261] T c25af37f2da84a3d85f92d1aa1a1e092 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Refusing update from remote peer e7ab6b9053664ee5aa71c7978e26fe6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:11.942373 32168 consensus_queue.cc:1035] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:11.942584 31957 consensus_queue.cc:1035] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:11.944779 31737 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:46:11.945175 29557 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:11.947906 29033 catalog_manager.cc:5576] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095) does not have the latest schema for tablet c25af37f2da84a3d85f92d1aa1a1e092 (table rebalancer_dynamic_table [id=ed288929859c426894df1426364b75f9]). Expected version 1 got 0
I20240502 20:46:11.955545 29039 catalog_manager.cc:5576] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751) does not have the latest schema for tablet c25af37f2da84a3d85f92d1aa1a1e092 (table rebalancer_dynamic_table [id=ed288929859c426894df1426364b75f9]). Expected version 1 got 0
I20240502 20:46:11.963696 32194 tablet.cc:1722] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:11.964007 32194 tablet.cc:1620] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:11.965691 29039 catalog_manager.cc:5576] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095) does not have the latest schema for tablet c25af37f2da84a3d85f92d1aa1a1e092 (table rebalancer_dynamic_table [id=ed288929859c426894df1426364b75f9]). Expected version 1 got 0
I20240502 20:46:11.966099 29033 catalog_manager.cc:5576] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751) does not have the latest schema for tablet c25af37f2da84a3d85f92d1aa1a1e092 (table rebalancer_dynamic_table [id=ed288929859c426894df1426364b75f9]). Expected version 1 got 0
I20240502 20:46:11.970611 32196 tablet.cc:1722] T c25af37f2da84a3d85f92d1aa1a1e092 P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:11.971244 29033 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=ed288929859c426894df1426364b75f9] alter complete (version 1)
I20240502 20:46:11.971339 32196 tablet.cc:1620] T c25af37f2da84a3d85f92d1aa1a1e092 P dbcdc6ac635b44b185c3eba63f66e9d6: MemRowSet was empty: no flush needed.
I20240502 20:46:11.980650 29424 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 20:46:11.982592 32195 tablet.cc:1722] T c25af37f2da84a3d85f92d1aa1a1e092 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:11.982888 32195 tablet.cc:1620] T c25af37f2da84a3d85f92d1aa1a1e092 P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:12.042407 29175 consensus_queue.cc:237] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:12.044013 29313 raft_consensus.cc:1261] T f19fa5a9a0e840c1bc0bc379a1907383 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.044072 29577 raft_consensus.cc:1261] T f19fa5a9a0e840c1bc0bc379a1907383 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.044373 32213 consensus_queue.cc:1035] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.044559 31396 consensus_queue.cc:1035] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.046994 31396 raft_consensus.cc:2932] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:12.047405 29313 raft_consensus.cc:2932] T f19fa5a9a0e840c1bc0bc379a1907383 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
W20240502 20:46:12.048333 29117 consensus_peers.cc:485] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d -> Peer e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): Couldn't send request to peer e7ab6b9053664ee5aa71c7978e26fe6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f19fa5a9a0e840c1bc0bc379a1907383. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:12.050386 29577 raft_consensus.cc:2932] T f19fa5a9a0e840c1bc0bc379a1907383 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:12.051031 29039 catalog_manager.cc:5484] T f19fa5a9a0e840c1bc0bc379a1907383 P 09258ac095664fa99caf0707262cde13 reported cstate change: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:12.061992 31956 consensus_queue.cc:1035] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:12.066251 31956 consensus_queue.cc:1035] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:12.070341 29313 consensus_queue.cc:237] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:12.071525 29175 raft_consensus.cc:1261] T 4bfb24d76b434cd5b138f868b1f29508 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.071612 29577 raft_consensus.cc:1261] T 4bfb24d76b434cd5b138f868b1f29508 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.071877 32191 consensus_queue.cc:1035] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 20:46:12.072046 29249 consensus_peers.cc:485] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4bfb24d76b434cd5b138f868b1f29508. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:12.072243 32191 consensus_queue.cc:1035] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.073427 29038 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:12.073865 32238 raft_consensus.cc:2932] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:12.074306 29577 raft_consensus.cc:2932] T 4bfb24d76b434cd5b138f868b1f29508 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:12.075811 29175 raft_consensus.cc:2932] T 4bfb24d76b434cd5b138f868b1f29508 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:12.079272 32194 tablet.cc:1722] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:12.079608 32194 tablet.cc:1620] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:12.079727 32196 tablet.cc:1722] T c25af37f2da84a3d85f92d1aa1a1e092 P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:12.077950 29033 catalog_manager.cc:5484] T 4bfb24d76b434cd5b138f868b1f29508 P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:12.080013 32196 tablet.cc:1620] T c25af37f2da84a3d85f92d1aa1a1e092 P dbcdc6ac635b44b185c3eba63f66e9d6: MemRowSet was empty: no flush needed.
I20240502 20:46:12.081561 32195 tablet.cc:1722] T c25af37f2da84a3d85f92d1aa1a1e092 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:12.081830 32195 tablet.cc:1620] T c25af37f2da84a3d85f92d1aa1a1e092 P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:12.084654 29039 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=ed288929859c426894df1426364b75f9] alter complete (version 2)
I20240502 20:46:12.098263 29175 consensus_queue.cc:237] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:12.099669 29313 raft_consensus.cc:1261] T 4fd651e80e004dc58d7769f227af58a8 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.099704 29446 raft_consensus.cc:1261] T 4fd651e80e004dc58d7769f227af58a8 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.100049 32217 consensus_queue.cc:1035] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.100247 32213 consensus_queue.cc:1035] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.102649 32213 raft_consensus.cc:2932] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
W20240502 20:46:12.103189 29119 consensus_peers.cc:485] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d -> Peer 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): Couldn't send request to peer 2a9c1af8cf814030bde56b67ca5c2fbe. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4fd651e80e004dc58d7769f227af58a8. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:12.103365 29446 raft_consensus.cc:2932] T 4fd651e80e004dc58d7769f227af58a8 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:12.104478 29313 raft_consensus.cc:2932] T 4fd651e80e004dc58d7769f227af58a8 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:12.106364 29038 catalog_manager.cc:5484] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d reported cstate change: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:12.124115 29446 consensus_queue.cc:237] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.125669 29313 raft_consensus.cc:1261] T 760d15b2acee4322a26f1f2a965f22cc P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.126031 29577 raft_consensus.cc:1261] T 760d15b2acee4322a26f1f2a965f22cc P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.126076 32073 consensus_queue.cc:1035] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.126399 32073 consensus_queue.cc:1035] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.128204 32275 raft_consensus.cc:2932] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:12.129158 29577 raft_consensus.cc:2932] T 760d15b2acee4322a26f1f2a965f22cc P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:12.129863 29313 raft_consensus.cc:2932] T 760d15b2acee4322a26f1f2a965f22cc P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
W20240502 20:46:12.131640 29380 consensus_peers.cc:485] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): Couldn't send request to peer e6b60803d0764d228c863e2da65e3691. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 760d15b2acee4322a26f1f2a965f22cc. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:12.131769 29033 catalog_manager.cc:5484] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:12.146431 29446 consensus_queue.cc:237] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:12.147636 29577 raft_consensus.cc:1261] T c18f9efa9c2043f6b2fad95ad9fd8296 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.147850 29313 raft_consensus.cc:1261] T c18f9efa9c2043f6b2fad95ad9fd8296 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.148020 31402 consensus_queue.cc:1035] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.148212 32275 consensus_queue.cc:1035] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.150027 31402 raft_consensus.cc:2932] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
W20240502 20:46:12.151089 29379 consensus_peers.cc:485] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): Couldn't send request to peer e7ab6b9053664ee5aa71c7978e26fe6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c18f9efa9c2043f6b2fad95ad9fd8296. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:12.151558 29577 raft_consensus.cc:2932] T c18f9efa9c2043f6b2fad95ad9fd8296 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:12.152225 29313 raft_consensus.cc:2932] T c18f9efa9c2043f6b2fad95ad9fd8296 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:12.152709 29039 catalog_manager.cc:5484] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:12.169813 29313 consensus_queue.cc:237] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:12.171165 29577 raft_consensus.cc:1261] T 2c6dfe6a92284c5f96094eb16c757222 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 20:46:12.171413 29249 consensus_peers.cc:485] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2c6dfe6a92284c5f96094eb16c757222. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:12.171617 32191 consensus_queue.cc:1035] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.174806 29175 raft_consensus.cc:1261] T 2c6dfe6a92284c5f96094eb16c757222 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.175165 32191 consensus_queue.cc:1035] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.176725 32238 raft_consensus.cc:2932] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:12.177119 29175 raft_consensus.cc:2932] T 2c6dfe6a92284c5f96094eb16c757222 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:12.178771 29577 raft_consensus.cc:2932] T 2c6dfe6a92284c5f96094eb16c757222 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:12.179827 29039 catalog_manager.cc:5484] T 2c6dfe6a92284c5f96094eb16c757222 P 7796ea3fba294d17a34386873969431d reported cstate change: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:12.186138 29038 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:12.191493 32194 tablet.cc:1722] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:12.192062 32196 tablet.cc:1722] T c25af37f2da84a3d85f92d1aa1a1e092 P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:12.195516 32195 tablet.cc:1722] T c25af37f2da84a3d85f92d1aa1a1e092 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:12.196736 29035 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=ed288929859c426894df1426364b75f9] alter complete (version 3)
I20240502 20:46:12.206226 29315 consensus_queue.cc:237] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:12.208055 29446 raft_consensus.cc:1261] T dcbca797b2d14e76b03e379cfe132597 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.208374 29577 raft_consensus.cc:1261] T dcbca797b2d14e76b03e379cfe132597 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.208472 32191 consensus_queue.cc:1035] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.208779 32191 consensus_queue.cc:1035] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.210355 32191 raft_consensus.cc:2932] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:12.211179 29577 raft_consensus.cc:2932] T dcbca797b2d14e76b03e379cfe132597 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:12.212100 29446 raft_consensus.cc:2932] T dcbca797b2d14e76b03e379cfe132597 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
W20240502 20:46:12.212359 29250 consensus_peers.cc:485] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 -> Peer 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): Couldn't send request to peer 2a9c1af8cf814030bde56b67ca5c2fbe. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dcbca797b2d14e76b03e379cfe132597. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:12.213002 29038 catalog_manager.cc:5484] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 reported cstate change: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:12.226979 32108 consensus_queue.cc:1035] T 6c29eeea8b9743e0a847a7cebc3c6e67 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:12.231863 32108 consensus_queue.cc:1035] T 6c29eeea8b9743e0a847a7cebc3c6e67 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:12.233697 29577 consensus_queue.cc:237] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.235054 29446 raft_consensus.cc:1261] T 94bdf135dd3a47d39e7b76dbfd524386 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.235432 29315 raft_consensus.cc:1261] T 94bdf135dd3a47d39e7b76dbfd524386 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.235461 32356 consensus_queue.cc:1035] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.235870 31187 consensus_queue.cc:1035] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.237737 31187 raft_consensus.cc:2932] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:12.238656 29315 raft_consensus.cc:2932] T 94bdf135dd3a47d39e7b76dbfd524386 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
W20240502 20:46:12.239017 29511 consensus_peers.cc:485] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057 -> Peer e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): Couldn't send request to peer e6b60803d0764d228c863e2da65e3691. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 94bdf135dd3a47d39e7b76dbfd524386. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:12.239579 29446 raft_consensus.cc:2932] T 94bdf135dd3a47d39e7b76dbfd524386 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:12.241299 29035 catalog_manager.cc:5484] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:12.257616 29315 consensus_queue.cc:237] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:12.258819 29175 raft_consensus.cc:1261] T 56928692a7284f33bd34e4396925c570 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.258935 29577 raft_consensus.cc:1261] T 56928692a7284f33bd34e4396925c570 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 20:46:12.259148 29248 consensus_peers.cc:485] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13 -> Peer e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): Couldn't send request to peer e7ab6b9053664ee5aa71c7978e26fe6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 56928692a7284f33bd34e4396925c570. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:12.259179 32315 consensus_queue.cc:1035] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.259336 32191 consensus_queue.cc:1035] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.261050 32191 raft_consensus.cc:2932] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:12.261446 29577 raft_consensus.cc:2932] T 56928692a7284f33bd34e4396925c570 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:12.263290 29175 raft_consensus.cc:2932] T 56928692a7284f33bd34e4396925c570 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:12.264052 29039 catalog_manager.cc:5484] T 56928692a7284f33bd34e4396925c570 P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:12.281495 29446 consensus_queue.cc:237] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:12.282788 29175 raft_consensus.cc:1261] T 846f1d6e39e642cfaabf96ad0c52366d P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.282989 29577 raft_consensus.cc:1261] T 846f1d6e39e642cfaabf96ad0c52366d P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.283179 32073 consensus_queue.cc:1035] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.283340 31402 consensus_queue.cc:1035] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.285096 31402 raft_consensus.cc:2932] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:12.285460 29577 raft_consensus.cc:2932] T 846f1d6e39e642cfaabf96ad0c52366d P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
W20240502 20:46:12.286233 29380 consensus_peers.cc:485] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 846f1d6e39e642cfaabf96ad0c52366d. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:12.286957 29175 raft_consensus.cc:2932] T 846f1d6e39e642cfaabf96ad0c52366d P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:12.287952 29035 catalog_manager.cc:5484] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:12.299327 29039 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:12.299432 29039 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:12.301342 29039 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet c25af37f2da84a3d85f92d1aa1a1e092
I20240502 20:46:12.301921 29424 tablet_service.cc:1514] Processing DeleteTablet for tablet c25af37f2da84a3d85f92d1aa1a1e092 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:12 UTC) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:12.302099 32091 tablet_replica.cc:330] T c25af37f2da84a3d85f92d1aa1a1e092 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:12.302220 32091 raft_consensus.cc:2229] T c25af37f2da84a3d85f92d1aa1a1e092 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:12.302299 32091 raft_consensus.cc:2258] T c25af37f2da84a3d85f92d1aa1a1e092 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:12.302414 31602 tablet_service.cc:1514] Processing DeleteTablet for tablet c25af37f2da84a3d85f92d1aa1a1e092 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:12 UTC) from {username='slave'} at 127.0.0.1:58346
I20240502 20:46:12.302410 31737 tablet_service.cc:1514] Processing DeleteTablet for tablet c25af37f2da84a3d85f92d1aa1a1e092 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:12 UTC) from {username='slave'} at 127.0.0.1:33864
I20240502 20:46:12.302740 32407 tablet_replica.cc:330] T c25af37f2da84a3d85f92d1aa1a1e092 P 315915efcf2b413ca7f9d874338e1b2d: stopping tablet replica
I20240502 20:46:12.302852 32407 raft_consensus.cc:2229] T c25af37f2da84a3d85f92d1aa1a1e092 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:12.302935 32091 ts_tablet_manager.cc:1875] T c25af37f2da84a3d85f92d1aa1a1e092 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:12.302960 32407 raft_consensus.cc:2258] T c25af37f2da84a3d85f92d1aa1a1e092 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:12.303191 32406 tablet_replica.cc:330] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a: stopping tablet replica
I20240502 20:46:12.303297 32406 raft_consensus.cc:2229] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:12.303584 32406 raft_consensus.cc:2258] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:12.303728 32407 ts_tablet_manager.cc:1875] T c25af37f2da84a3d85f92d1aa1a1e092 P 315915efcf2b413ca7f9d874338e1b2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:12.304190 32406 ts_tablet_manager.cc:1875] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:12.304993 32091 ts_tablet_manager.cc:1888] T c25af37f2da84a3d85f92d1aa1a1e092 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:12.305056 32091 log.cc:1196] T c25af37f2da84a3d85f92d1aa1a1e092 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/c25af37f2da84a3d85f92d1aa1a1e092
I20240502 20:46:12.305299 32091 ts_tablet_manager.cc:1909] T c25af37f2da84a3d85f92d1aa1a1e092 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting consensus metadata
I20240502 20:46:12.305836 32407 ts_tablet_manager.cc:1888] T c25af37f2da84a3d85f92d1aa1a1e092 P 315915efcf2b413ca7f9d874338e1b2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:12.305902 32407 log.cc:1196] T c25af37f2da84a3d85f92d1aa1a1e092 P 315915efcf2b413ca7f9d874338e1b2d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal/wals/c25af37f2da84a3d85f92d1aa1a1e092
I20240502 20:46:12.306141 32407 ts_tablet_manager.cc:1909] T c25af37f2da84a3d85f92d1aa1a1e092 P 315915efcf2b413ca7f9d874338e1b2d: Deleting consensus metadata
I20240502 20:46:12.306217 32406 ts_tablet_manager.cc:1888] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:12.306293 32406 log.cc:1196] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/wals/c25af37f2da84a3d85f92d1aa1a1e092
I20240502 20:46:12.306313 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet c25af37f2da84a3d85f92d1aa1a1e092 (table rebalancer_dynamic_table_new_name [id=ed288929859c426894df1426364b75f9]) successfully deleted
I20240502 20:46:12.306614 32406 ts_tablet_manager.cc:1909] T c25af37f2da84a3d85f92d1aa1a1e092 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting consensus metadata
I20240502 20:46:12.306852 29027 catalog_manager.cc:4830] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): tablet c25af37f2da84a3d85f92d1aa1a1e092 (table rebalancer_dynamic_table_new_name [id=ed288929859c426894df1426364b75f9]) successfully deleted
I20240502 20:46:12.307283 29026 catalog_manager.cc:4830] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): tablet c25af37f2da84a3d85f92d1aa1a1e092 (table rebalancer_dynamic_table_new_name [id=ed288929859c426894df1426364b75f9]) successfully deleted
I20240502 20:46:12.309873 29175 consensus_queue.cc:237] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:12.311266 29313 raft_consensus.cc:1261] T d167293496fb4334ae670a62a2f80fa9 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.311288 29446 raft_consensus.cc:1261] T d167293496fb4334ae670a62a2f80fa9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 20:46:12.311491 29119 consensus_peers.cc:485] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d -> Peer 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): Couldn't send request to peer 2a9c1af8cf814030bde56b67ca5c2fbe. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d167293496fb4334ae670a62a2f80fa9. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:12.311714 32217 consensus_queue.cc:1035] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.311884 32213 consensus_queue.cc:1035] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.313545 32217 raft_consensus.cc:2932] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:12.313987 29313 raft_consensus.cc:2932] T d167293496fb4334ae670a62a2f80fa9 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:12.315380 29446 raft_consensus.cc:2932] T d167293496fb4334ae670a62a2f80fa9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:12.317719 29039 catalog_manager.cc:5484] T d167293496fb4334ae670a62a2f80fa9 P 09258ac095664fa99caf0707262cde13 reported cstate change: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:12.334138 29446 consensus_queue.cc:237] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.335394 29313 raft_consensus.cc:1261] T 62f3301937ba4471afcc327be05486cf P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.335489 29175 raft_consensus.cc:1261] T 62f3301937ba4471afcc327be05486cf P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.335917 32073 consensus_queue.cc:1035] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.336423 32073 consensus_queue.cc:1035] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.337954 32073 raft_consensus.cc:2932] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:12.338395 29313 raft_consensus.cc:2932] T 62f3301937ba4471afcc327be05486cf P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
W20240502 20:46:12.339398 29380 consensus_peers.cc:485] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): Couldn't send request to peer e6b60803d0764d228c863e2da65e3691. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 62f3301937ba4471afcc327be05486cf. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:12.339731 29182 raft_consensus.cc:2932] T 62f3301937ba4471afcc327be05486cf P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:12.340525 29035 catalog_manager.cc:5484] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:12.363482 29446 consensus_queue.cc:237] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
W20240502 20:46:12.365861 29379 consensus_peers.cc:485] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): Couldn't send request to peer e7ab6b9053664ee5aa71c7978e26fe6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a8916d4762844444bae6fa6f930abbfd. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:12.366000 29577 raft_consensus.cc:1261] T a8916d4762844444bae6fa6f930abbfd P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.366202 29182 raft_consensus.cc:1261] T a8916d4762844444bae6fa6f930abbfd P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.366442 31402 consensus_queue.cc:1035] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.366647 32275 consensus_queue.cc:1035] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.368991 32275 raft_consensus.cc:2932] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:12.370704 29182 raft_consensus.cc:2932] T a8916d4762844444bae6fa6f930abbfd P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:12.371585 29039 catalog_manager.cc:5484] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:12.374627 29577 raft_consensus.cc:2932] T a8916d4762844444bae6fa6f930abbfd P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:12.385164 32172 consensus_queue.cc:1035] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:12.389613 32111 consensus_queue.cc:1035] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:12.395478 29446 consensus_queue.cc:237] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:12.396667 29182 raft_consensus.cc:1261] T 2d72ef6e7e8d473baaff8f527c02332b P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.396697 29577 raft_consensus.cc:1261] T 2d72ef6e7e8d473baaff8f527c02332b P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.397004 32073 consensus_queue.cc:1035] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.397171 31402 consensus_queue.cc:1035] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 20:46:12.397346 29380 consensus_peers.cc:485] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2d72ef6e7e8d473baaff8f527c02332b. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:12.398953 31402 raft_consensus.cc:2932] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:12.399308 29577 raft_consensus.cc:2932] T 2d72ef6e7e8d473baaff8f527c02332b P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:12.400439 29182 raft_consensus.cc:2932] T 2d72ef6e7e8d473baaff8f527c02332b P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:12.402694 29039 catalog_manager.cc:5484] T 2d72ef6e7e8d473baaff8f527c02332b P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:12.403993 29033 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:12.406301 29035 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalance_test_table_4" } modify_external_catalogs: true
I20240502 20:46:12.406414 29035 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalance_test_table_4" } modify_external_catalogs: true
I20240502 20:46:12.409070 29035 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 44010b796bae4b97ab7f17ee5cdffc30
I20240502 20:46:12.409288 31475 tablet_service.cc:1467] Processing CreateTablet for tablet d1546d04e9e844378e6a8ed2f32e3d31 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=e52fed40cee049f4a63723d5f76fb1a3]), partition=
I20240502 20:46:12.409372 29035 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 6a3c10a388d8469bb04a21b48b4aafaa
I20240502 20:46:12.409541 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1546d04e9e844378e6a8ed2f32e3d31. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:12.409587 29035 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet fbd8fbbb18334242834ec7fa39184204
I20240502 20:46:12.410120 29555 tablet_service.cc:1514] Processing DeleteTablet for tablet fbd8fbbb18334242834ec7fa39184204 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:12 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:12.410226 29164 tablet_service.cc:1514] Processing DeleteTablet for tablet 44010b796bae4b97ab7f17ee5cdffc30 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:12 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:12.410218 29295 tablet_service.cc:1514] Processing DeleteTablet for tablet 44010b796bae4b97ab7f17ee5cdffc30 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:12 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:12.410377 29557 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a3c10a388d8469bb04a21b48b4aafaa with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:12 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:12.410514 32477 tablet_replica.cc:330] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:12.410665 32477 raft_consensus.cc:2229] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:12.410892 32477 raft_consensus.cc:2258] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:12.410954 32089 tablet_replica.cc:330] T 44010b796bae4b97ab7f17ee5cdffc30 P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:12.411083 32089 raft_consensus.cc:2229] T 44010b796bae4b97ab7f17ee5cdffc30 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:12.411214 32089 raft_consensus.cc:2258] T 44010b796bae4b97ab7f17ee5cdffc30 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:12.411412 32477 ts_tablet_manager.cc:1875] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:12.411692 32089 ts_tablet_manager.cc:1875] T 44010b796bae4b97ab7f17ee5cdffc30 P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:12.411945 29424 tablet_service.cc:1514] Processing DeleteTablet for tablet 44010b796bae4b97ab7f17ee5cdffc30 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:12 UTC) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:12.412102 32088 tablet_replica.cc:330] T fbd8fbbb18334242834ec7fa39184204 P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:12.412235 32088 raft_consensus.cc:2229] T fbd8fbbb18334242834ec7fa39184204 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:12.412371 32088 raft_consensus.cc:2258] T fbd8fbbb18334242834ec7fa39184204 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:12.412875 32088 ts_tablet_manager.cc:1875] T fbd8fbbb18334242834ec7fa39184204 P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:12.413590 32477 ts_tablet_manager.cc:1888] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:12.413653 32477 log.cc:1196] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/44010b796bae4b97ab7f17ee5cdffc30
I20240502 20:46:12.413802 32089 ts_tablet_manager.cc:1888] T 44010b796bae4b97ab7f17ee5cdffc30 P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:12.413864 32089 log.cc:1196] T 44010b796bae4b97ab7f17ee5cdffc30 P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/44010b796bae4b97ab7f17ee5cdffc30
I20240502 20:46:12.413897 32477 ts_tablet_manager.cc:1909] T 44010b796bae4b97ab7f17ee5cdffc30 P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:12.414074 32089 ts_tablet_manager.cc:1909] T 44010b796bae4b97ab7f17ee5cdffc30 P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:12.414676 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 44010b796bae4b97ab7f17ee5cdffc30 (table rebalance_test_table_4 [id=b684c3e68fbe4841bf5ca5674beb20f1]) successfully deleted
I20240502 20:46:12.414741 31737 tablet_service.cc:1467] Processing CreateTablet for tablet d1546d04e9e844378e6a8ed2f32e3d31 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=e52fed40cee049f4a63723d5f76fb1a3]), partition=
I20240502 20:46:12.414765 29164 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a3c10a388d8469bb04a21b48b4aafaa with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:12 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:12.414835 29163 tablet_service.cc:1514] Processing DeleteTablet for tablet fbd8fbbb18334242834ec7fa39184204 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:12 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:12.414906 31602 tablet_service.cc:1467] Processing CreateTablet for tablet d1546d04e9e844378e6a8ed2f32e3d31 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=e52fed40cee049f4a63723d5f76fb1a3]), partition=
I20240502 20:46:12.414983 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1546d04e9e844378e6a8ed2f32e3d31. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:12.415094 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1546d04e9e844378e6a8ed2f32e3d31. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:12.416498 32484 tablet_bootstrap.cc:492] T d1546d04e9e844378e6a8ed2f32e3d31 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:12.416515 32091 tablet_replica.cc:330] T 44010b796bae4b97ab7f17ee5cdffc30 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:12.416638 32091 raft_consensus.cc:2229] T 44010b796bae4b97ab7f17ee5cdffc30 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:12.416767 32091 raft_consensus.cc:2258] T 44010b796bae4b97ab7f17ee5cdffc30 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:12.416931 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet 44010b796bae4b97ab7f17ee5cdffc30 (table rebalance_test_table_4 [id=b684c3e68fbe4841bf5ca5674beb20f1]) successfully deleted
I20240502 20:46:12.416958 29295 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a3c10a388d8469bb04a21b48b4aafaa with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:12 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:12.417263 32091 ts_tablet_manager.cc:1875] T 44010b796bae4b97ab7f17ee5cdffc30 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:12.417479 32089 tablet_replica.cc:330] T 6a3c10a388d8469bb04a21b48b4aafaa P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:12.417587 32484 tablet_bootstrap.cc:654] T d1546d04e9e844378e6a8ed2f32e3d31 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:12.417616 32089 raft_consensus.cc:2229] T 6a3c10a388d8469bb04a21b48b4aafaa P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:12.417773 32089 raft_consensus.cc:2258] T 6a3c10a388d8469bb04a21b48b4aafaa P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:12.418094 32477 tablet_replica.cc:330] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:12.418159 32089 ts_tablet_manager.cc:1875] T 6a3c10a388d8469bb04a21b48b4aafaa P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:12.418226 32477 raft_consensus.cc:2229] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:12.418392 32477 raft_consensus.cc:2258] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:12.418911 32477 ts_tablet_manager.cc:1875] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:12.419299 32091 ts_tablet_manager.cc:1888] T 44010b796bae4b97ab7f17ee5cdffc30 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:12.419356 32091 log.cc:1196] T 44010b796bae4b97ab7f17ee5cdffc30 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/44010b796bae4b97ab7f17ee5cdffc30
I20240502 20:46:12.419574 32091 ts_tablet_manager.cc:1909] T 44010b796bae4b97ab7f17ee5cdffc30 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting consensus metadata
I20240502 20:46:12.419793 32089 ts_tablet_manager.cc:1888] T 6a3c10a388d8469bb04a21b48b4aafaa P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:12.419853 32089 log.cc:1196] T 6a3c10a388d8469bb04a21b48b4aafaa P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/6a3c10a388d8469bb04a21b48b4aafaa
I20240502 20:46:12.420087 32089 ts_tablet_manager.cc:1909] T 6a3c10a388d8469bb04a21b48b4aafaa P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:12.420197 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet 44010b796bae4b97ab7f17ee5cdffc30 (table rebalance_test_table_4 [id=b684c3e68fbe4841bf5ca5674beb20f1]) successfully deleted
I20240502 20:46:12.420725 32477 ts_tablet_manager.cc:1888] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:12.420797 32477 log.cc:1196] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/6a3c10a388d8469bb04a21b48b4aafaa
I20240502 20:46:12.420949 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 6a3c10a388d8469bb04a21b48b4aafaa (table rebalance_test_table_4 [id=b684c3e68fbe4841bf5ca5674beb20f1]) successfully deleted
I20240502 20:46:12.421016 32477 ts_tablet_manager.cc:1909] T 6a3c10a388d8469bb04a21b48b4aafaa P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:12.421137 32089 tablet_replica.cc:330] T fbd8fbbb18334242834ec7fa39184204 P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:12.421499 29424 tablet_service.cc:1514] Processing DeleteTablet for tablet fbd8fbbb18334242834ec7fa39184204 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:12 UTC) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:12.421572 32089 raft_consensus.cc:2229] T fbd8fbbb18334242834ec7fa39184204 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:12.421591 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet 6a3c10a388d8469bb04a21b48b4aafaa (table rebalance_test_table_4 [id=b684c3e68fbe4841bf5ca5674beb20f1]) successfully deleted
I20240502 20:46:12.421707 32089 raft_consensus.cc:2258] T fbd8fbbb18334242834ec7fa39184204 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:12.422135 32089 ts_tablet_manager.cc:1875] T fbd8fbbb18334242834ec7fa39184204 P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:12.422317 32088 ts_tablet_manager.cc:1888] T fbd8fbbb18334242834ec7fa39184204 P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:12.422376 32088 log.cc:1196] T fbd8fbbb18334242834ec7fa39184204 P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/fbd8fbbb18334242834ec7fa39184204
I20240502 20:46:12.422571 32091 tablet_replica.cc:330] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:12.422658 32088 ts_tablet_manager.cc:1909] T fbd8fbbb18334242834ec7fa39184204 P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:12.422731 32091 raft_consensus.cc:2229] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:12.422899 32091 raft_consensus.cc:2258] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:12.422646 32485 tablet_bootstrap.cc:492] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:12.423287 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet fbd8fbbb18334242834ec7fa39184204 (table rebalance_test_table_4 [id=b684c3e68fbe4841bf5ca5674beb20f1]) successfully deleted
I20240502 20:46:12.423321 32091 ts_tablet_manager.cc:1875] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:12.423358 32088 tablet_replica.cc:330] T 6a3c10a388d8469bb04a21b48b4aafaa P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:12.423556 32088 raft_consensus.cc:2229] T 6a3c10a388d8469bb04a21b48b4aafaa P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:12.423712 32088 raft_consensus.cc:2258] T 6a3c10a388d8469bb04a21b48b4aafaa P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:12.424067 32089 ts_tablet_manager.cc:1888] T fbd8fbbb18334242834ec7fa39184204 P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:12.424187 32089 log.cc:1196] T fbd8fbbb18334242834ec7fa39184204 P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/fbd8fbbb18334242834ec7fa39184204
I20240502 20:46:12.424481 32089 ts_tablet_manager.cc:1909] T fbd8fbbb18334242834ec7fa39184204 P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:12.424563 32485 tablet_bootstrap.cc:654] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:12.425005 32091 ts_tablet_manager.cc:1888] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:12.425100 32091 log.cc:1196] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/fbd8fbbb18334242834ec7fa39184204
I20240502 20:46:12.425100 32088 ts_tablet_manager.cc:1875] T 6a3c10a388d8469bb04a21b48b4aafaa P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:12.425128 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet fbd8fbbb18334242834ec7fa39184204 (table rebalance_test_table_4 [id=b684c3e68fbe4841bf5ca5674beb20f1]) successfully deleted
I20240502 20:46:12.425375 32091 ts_tablet_manager.cc:1909] T fbd8fbbb18334242834ec7fa39184204 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting consensus metadata
I20240502 20:46:12.425982 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet fbd8fbbb18334242834ec7fa39184204 (table rebalance_test_table_4 [id=b684c3e68fbe4841bf5ca5674beb20f1]) successfully deleted
I20240502 20:46:12.426167 32478 tablet_bootstrap.cc:492] T d1546d04e9e844378e6a8ed2f32e3d31 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:12.426944 32088 ts_tablet_manager.cc:1888] T 6a3c10a388d8469bb04a21b48b4aafaa P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:12.427062 32088 log.cc:1196] T 6a3c10a388d8469bb04a21b48b4aafaa P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/6a3c10a388d8469bb04a21b48b4aafaa
I20240502 20:46:12.427073 32484 tablet_bootstrap.cc:492] T d1546d04e9e844378e6a8ed2f32e3d31 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:12.427175 32478 tablet_bootstrap.cc:654] T d1546d04e9e844378e6a8ed2f32e3d31 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:12.427395 32088 ts_tablet_manager.cc:1909] T 6a3c10a388d8469bb04a21b48b4aafaa P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:12.427999 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet 6a3c10a388d8469bb04a21b48b4aafaa (table rebalance_test_table_4 [id=b684c3e68fbe4841bf5ca5674beb20f1]) successfully deleted
I20240502 20:46:12.427282 32484 ts_tablet_manager.cc:1367] T d1546d04e9e844378e6a8ed2f32e3d31 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 20:46:12.428198 32478 tablet_bootstrap.cc:492] T d1546d04e9e844378e6a8ed2f32e3d31 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:12.428279 32478 ts_tablet_manager.cc:1367] T d1546d04e9e844378e6a8ed2f32e3d31 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:12.428489 32485 tablet_bootstrap.cc:492] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:12.428567 32485 ts_tablet_manager.cc:1367] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 20:46:12.428587 32478 raft_consensus.cc:348] T d1546d04e9e844378e6a8ed2f32e3d31 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:12.428575 32484 raft_consensus.cc:348] T d1546d04e9e844378e6a8ed2f32e3d31 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:12.428660 32478 raft_consensus.cc:374] T d1546d04e9e844378e6a8ed2f32e3d31 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:12.428679 32478 raft_consensus.cc:729] T d1546d04e9e844378e6a8ed2f32e3d31 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:12.428673 32484 raft_consensus.cc:374] T d1546d04e9e844378e6a8ed2f32e3d31 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:12.428707 32484 raft_consensus.cc:729] T d1546d04e9e844378e6a8ed2f32e3d31 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:12.428781 32478 consensus_queue.cc:260] T d1546d04e9e844378e6a8ed2f32e3d31 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:12.428822 32484 consensus_queue.cc:260] T d1546d04e9e844378e6a8ed2f32e3d31 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:12.429033 32478 ts_tablet_manager.cc:1398] T d1546d04e9e844378e6a8ed2f32e3d31 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:12.428992 32485 raft_consensus.cc:348] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:12.429082 32485 raft_consensus.cc:374] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:12.429113 32485 raft_consensus.cc:729] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:12.429231 32485 consensus_queue.cc:260] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:12.429435 32484 ts_tablet_manager.cc:1398] T d1546d04e9e844378e6a8ed2f32e3d31 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:12.429461 32485 ts_tablet_manager.cc:1398] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:12.438926 29182 consensus_queue.cc:237] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.440187 29446 raft_consensus.cc:1261] T 616e6b2cacc54145a29a10cc8584bbdf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.440384 29577 raft_consensus.cc:1261] T 616e6b2cacc54145a29a10cc8584bbdf P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.440551 32217 consensus_queue.cc:1035] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.440837 32213 consensus_queue.cc:1035] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.442842 32217 raft_consensus.cc:2932] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:12.444454 29446 raft_consensus.cc:2932] T 616e6b2cacc54145a29a10cc8584bbdf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
W20240502 20:46:12.444792 29118 consensus_peers.cc:485] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d -> Peer e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): Couldn't send request to peer e6b60803d0764d228c863e2da65e3691. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 616e6b2cacc54145a29a10cc8584bbdf. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:12.444680 29577 raft_consensus.cc:2932] T 616e6b2cacc54145a29a10cc8584bbdf P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:12.446522 29042 catalog_manager.cc:5484] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d reported cstate change: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:12.462893 29182 consensus_queue.cc:237] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:12.464159 29577 raft_consensus.cc:1261] T e01c5b97e2d14976812691b78a5143c5 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.464361 29313 raft_consensus.cc:1261] T e01c5b97e2d14976812691b78a5143c5 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 20:46:12.464500 29119 consensus_peers.cc:485] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d -> Peer 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): Couldn't send request to peer 2a9c1af8cf814030bde56b67ca5c2fbe. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e01c5b97e2d14976812691b78a5143c5. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:12.464546 32213 consensus_queue.cc:1035] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.464776 31397 consensus_queue.cc:1035] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.466567 32213 raft_consensus.cc:2932] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:12.466966 29313 raft_consensus.cc:2932] T e01c5b97e2d14976812691b78a5143c5 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:12.469839 29038 catalog_manager.cc:5484] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d reported cstate change: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:12.471194 29577 raft_consensus.cc:2932] T e01c5b97e2d14976812691b78a5143c5 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:12.487057 32530 ts_tablet_manager.cc:902] T 4bfb24d76b434cd5b138f868b1f29508 P 315915efcf2b413ca7f9d874338e1b2d: Initiating tablet copy from peer 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:12.487630 29446 consensus_queue.cc:237] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:12.487910 32530 tablet_copy_client.cc:308] T 4bfb24d76b434cd5b138f868b1f29508 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.66:40945
I20240502 20:46:12.489409 29313 raft_consensus.cc:1261] T 9d2cd90615dc4d1894ffe5e80914bdc0 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.489495 29577 raft_consensus.cc:1261] T 9d2cd90615dc4d1894ffe5e80914bdc0 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 20:46:12.489763 29379 consensus_peers.cc:485] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): Couldn't send request to peer e7ab6b9053664ee5aa71c7978e26fe6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9d2cd90615dc4d1894ffe5e80914bdc0. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:12.489830 32073 consensus_queue.cc:1035] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.489984 31402 consensus_queue.cc:1035] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.492445 29335 tablet_copy_service.cc:140] P 09258ac095664fa99caf0707262cde13: Received BeginTabletCopySession request for tablet 4bfb24d76b434cd5b138f868b1f29508 from peer 315915efcf2b413ca7f9d874338e1b2d ({username='slave'} at 127.22.45.70:37603)
I20240502 20:46:12.492558 29335 tablet_copy_service.cc:161] P 09258ac095664fa99caf0707262cde13: Beginning new tablet copy session on tablet 4bfb24d76b434cd5b138f868b1f29508 from peer 315915efcf2b413ca7f9d874338e1b2d at {username='slave'} at 127.22.45.70:37603: session id = 315915efcf2b413ca7f9d874338e1b2d-4bfb24d76b434cd5b138f868b1f29508
I20240502 20:46:12.492491 32073 raft_consensus.cc:2932] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:12.493816 29335 tablet_copy_source_session.cc:205] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:12.494282 29313 raft_consensus.cc:2932] T 9d2cd90615dc4d1894ffe5e80914bdc0 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:12.494843 29577 raft_consensus.cc:2932] T 9d2cd90615dc4d1894ffe5e80914bdc0 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:12.495519 29038 catalog_manager.cc:5484] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:12.496735 32530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bfb24d76b434cd5b138f868b1f29508. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:12.499706 29035 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:12.499811 32530 tablet_copy_client.cc:783] T 4bfb24d76b434cd5b138f868b1f29508 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:12.499981 32530 tablet_copy_client.cc:647] T 4bfb24d76b434cd5b138f868b1f29508 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:12.502944 32530 tablet_copy_client.cc:523] T 4bfb24d76b434cd5b138f868b1f29508 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:12.504464 32530 tablet_bootstrap.cc:492] T 4bfb24d76b434cd5b138f868b1f29508 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:12.505873 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 90bab3d11d424b3abcdc74d5da51cbee (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=536c055ebf024097b825854cd0490c67]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:12.506186 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90bab3d11d424b3abcdc74d5da51cbee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:12.506202 31737 tablet_service.cc:1467] Processing CreateTablet for tablet 90bab3d11d424b3abcdc74d5da51cbee (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=536c055ebf024097b825854cd0490c67]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:12.506366 31602 tablet_service.cc:1467] Processing CreateTablet for tablet 90bab3d11d424b3abcdc74d5da51cbee (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=536c055ebf024097b825854cd0490c67]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:12.506594 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90bab3d11d424b3abcdc74d5da51cbee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:12.506628 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90bab3d11d424b3abcdc74d5da51cbee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:12.506667 29163 tablet_service.cc:1467] Processing CreateTablet for tablet 075d3240e24a4532bbb29a068c00c128 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=536c055ebf024097b825854cd0490c67]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:12.506916 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 075d3240e24a4532bbb29a068c00c128. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:12.508438 32485 tablet_bootstrap.cc:492] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:12.508888 32484 tablet_bootstrap.cc:492] T 90bab3d11d424b3abcdc74d5da51cbee P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:12.509338 32485 tablet_bootstrap.cc:654] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:12.509743 32484 tablet_bootstrap.cc:654] T 90bab3d11d424b3abcdc74d5da51cbee P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:12.509768 31475 tablet_service.cc:1467] Processing CreateTablet for tablet 0f57649f122e45f9b8ef82ec975e5359 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=536c055ebf024097b825854cd0490c67]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:12.510043 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f57649f122e45f9b8ef82ec975e5359. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:12.510726 32545 tablet_bootstrap.cc:492] T 075d3240e24a4532bbb29a068c00c128 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:12.510756 32485 tablet_bootstrap.cc:492] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:12.510761 31736 tablet_service.cc:1467] Processing CreateTablet for tablet 075d3240e24a4532bbb29a068c00c128 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=536c055ebf024097b825854cd0490c67]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:12.510828 32485 ts_tablet_manager.cc:1367] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:12.511006 31736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 075d3240e24a4532bbb29a068c00c128. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:12.511269 32485 raft_consensus.cc:348] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:12.511369 32485 raft_consensus.cc:374] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:12.511399 32485 raft_consensus.cc:729] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:12.511504 32485 consensus_queue.cc:260] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:12.511775 32485 ts_tablet_manager.cc:1398] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:12.511817 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 075d3240e24a4532bbb29a068c00c128 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=536c055ebf024097b825854cd0490c67]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:12.512065 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 075d3240e24a4532bbb29a068c00c128. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:12.512761 31602 tablet_service.cc:1467] Processing CreateTablet for tablet 0f57649f122e45f9b8ef82ec975e5359 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=536c055ebf024097b825854cd0490c67]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:12.512974 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f57649f122e45f9b8ef82ec975e5359. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:12.514135 32478 tablet_bootstrap.cc:492] T 0f57649f122e45f9b8ef82ec975e5359 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:12.514106 31868 tablet_service.cc:1467] Processing CreateTablet for tablet 0f57649f122e45f9b8ef82ec975e5359 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=536c055ebf024097b825854cd0490c67]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:12.514336 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f57649f122e45f9b8ef82ec975e5359. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:12.514931 32478 tablet_bootstrap.cc:654] T 0f57649f122e45f9b8ef82ec975e5359 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:12.515678 32485 tablet_bootstrap.cc:492] T 0f57649f122e45f9b8ef82ec975e5359 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:12.516510 32485 tablet_bootstrap.cc:654] T 0f57649f122e45f9b8ef82ec975e5359 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:12.516963 32545 tablet_bootstrap.cc:654] T 075d3240e24a4532bbb29a068c00c128 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:12.517745 32544 tablet_bootstrap.cc:492] T 90bab3d11d424b3abcdc74d5da51cbee P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:12.518066 32484 tablet_bootstrap.cc:492] T 90bab3d11d424b3abcdc74d5da51cbee P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:12.518150 32484 ts_tablet_manager.cc:1367] T 90bab3d11d424b3abcdc74d5da51cbee P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 20:46:12.518822 32544 tablet_bootstrap.cc:654] T 90bab3d11d424b3abcdc74d5da51cbee P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:12.518916 32484 raft_consensus.cc:348] T 90bab3d11d424b3abcdc74d5da51cbee P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:12.519018 32484 raft_consensus.cc:374] T 90bab3d11d424b3abcdc74d5da51cbee P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:12.519052 32484 raft_consensus.cc:729] T 90bab3d11d424b3abcdc74d5da51cbee P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:12.519193 32484 consensus_queue.cc:260] T 90bab3d11d424b3abcdc74d5da51cbee P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:12.519464 32484 ts_tablet_manager.cc:1398] T 90bab3d11d424b3abcdc74d5da51cbee P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:12.519625 32484 tablet_bootstrap.cc:492] T 075d3240e24a4532bbb29a068c00c128 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:12.519932 32485 tablet_bootstrap.cc:492] T 0f57649f122e45f9b8ef82ec975e5359 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:12.520005 32485 ts_tablet_manager.cc:1367] T 0f57649f122e45f9b8ef82ec975e5359 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:12.520397 32485 raft_consensus.cc:348] T 0f57649f122e45f9b8ef82ec975e5359 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.520471 32484 tablet_bootstrap.cc:654] T 075d3240e24a4532bbb29a068c00c128 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:12.520489 32485 raft_consensus.cc:374] T 0f57649f122e45f9b8ef82ec975e5359 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:12.520522 32485 raft_consensus.cc:729] T 0f57649f122e45f9b8ef82ec975e5359 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:12.520632 32485 consensus_queue.cc:260] T 0f57649f122e45f9b8ef82ec975e5359 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.520887 32485 ts_tablet_manager.cc:1398] T 0f57649f122e45f9b8ef82ec975e5359 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:12.520973 32478 tablet_bootstrap.cc:492] T 0f57649f122e45f9b8ef82ec975e5359 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:12.521060 32478 ts_tablet_manager.cc:1367] T 0f57649f122e45f9b8ef82ec975e5359 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 20:46:12.521790 32478 raft_consensus.cc:348] T 0f57649f122e45f9b8ef82ec975e5359 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.521884 32478 raft_consensus.cc:374] T 0f57649f122e45f9b8ef82ec975e5359 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:12.521910 32478 raft_consensus.cc:729] T 0f57649f122e45f9b8ef82ec975e5359 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:12.522017 32478 consensus_queue.cc:260] T 0f57649f122e45f9b8ef82ec975e5359 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.522264 32478 ts_tablet_manager.cc:1398] T 0f57649f122e45f9b8ef82ec975e5359 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:12.522292 32484 tablet_bootstrap.cc:492] T 075d3240e24a4532bbb29a068c00c128 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:12.522365 32484 ts_tablet_manager.cc:1367] T 075d3240e24a4532bbb29a068c00c128 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 20:46:12.522881 32484 raft_consensus.cc:348] T 075d3240e24a4532bbb29a068c00c128 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.522969 32484 raft_consensus.cc:374] T 075d3240e24a4532bbb29a068c00c128 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:12.522996 32484 raft_consensus.cc:729] T 075d3240e24a4532bbb29a068c00c128 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:12.523105 32484 consensus_queue.cc:260] T 075d3240e24a4532bbb29a068c00c128 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.523201 32544 tablet_bootstrap.cc:492] T 90bab3d11d424b3abcdc74d5da51cbee P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:12.523272 32544 ts_tablet_manager.cc:1367] T 90bab3d11d424b3abcdc74d5da51cbee P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 20:46:12.523344 32484 ts_tablet_manager.cc:1398] T 075d3240e24a4532bbb29a068c00c128 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:12.523641 32544 raft_consensus.cc:348] T 90bab3d11d424b3abcdc74d5da51cbee P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:12.523736 32544 raft_consensus.cc:374] T 90bab3d11d424b3abcdc74d5da51cbee P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:12.523761 32544 raft_consensus.cc:729] T 90bab3d11d424b3abcdc74d5da51cbee P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:12.523867 32544 consensus_queue.cc:260] T 90bab3d11d424b3abcdc74d5da51cbee P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:12.526059 32544 ts_tablet_manager.cc:1398] T 90bab3d11d424b3abcdc74d5da51cbee P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:12.526293 32544 tablet_bootstrap.cc:492] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:12.526523 31956 raft_consensus.cc:482] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:12.526595 31956 raft_consensus.cc:504] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:12.527009 31956 leader_election.cc:288] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:12.530041 32544 tablet_bootstrap.cc:654] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:12.531466 32544 tablet_bootstrap.cc:492] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:12.531538 32544 ts_tablet_manager.cc:1367] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 20:46:12.531920 32544 raft_consensus.cc:348] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.532011 32544 raft_consensus.cc:374] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:12.532037 32544 raft_consensus.cc:729] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:12.532135 32544 consensus_queue.cc:260] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.532370 32544 ts_tablet_manager.cc:1398] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:12.532500 32544 tablet_bootstrap.cc:492] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:12.532718 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1546d04e9e844378e6a8ed2f32e3d31" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:12.532858 31757 raft_consensus.cc:2454] T d1546d04e9e844378e6a8ed2f32e3d31 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:12.533140 31559 leader_election.cc:302] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:12.533537 32544 tablet_bootstrap.cc:654] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:12.533982 31956 raft_consensus.cc:2781] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:12.534034 31956 raft_consensus.cc:482] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:12.534066 31956 raft_consensus.cc:3037] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:12.535024 31956 raft_consensus.cc:504] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:12.535331 31956 leader_election.cc:288] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Requested vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:12.535588 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1546d04e9e844378e6a8ed2f32e3d31" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:12.535681 31757 raft_consensus.cc:3037] T d1546d04e9e844378e6a8ed2f32e3d31 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:12.536237 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1546d04e9e844378e6a8ed2f32e3d31" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:12.536377 31495 raft_consensus.cc:2454] T d1546d04e9e844378e6a8ed2f32e3d31 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:12.536558 31757 raft_consensus.cc:2454] T d1546d04e9e844378e6a8ed2f32e3d31 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:12.536569 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1546d04e9e844378e6a8ed2f32e3d31" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:12.536649 31494 raft_consensus.cc:3037] T d1546d04e9e844378e6a8ed2f32e3d31 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:12.536829 31559 leader_election.cc:302] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:12.537559 31494 raft_consensus.cc:2454] T d1546d04e9e844378e6a8ed2f32e3d31 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:12.538806 31956 raft_consensus.cc:2781] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:12.538887 31956 raft_consensus.cc:686] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Becoming Leader. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Running, Role: LEADER
I20240502 20:46:12.538991 31956 consensus_queue.cc:237] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:12.540541 29035 catalog_manager.cc:5484] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70). New cstate: current_term: 1 leader_uuid: "315915efcf2b413ca7f9d874338e1b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:12.541143 32544 tablet_bootstrap.cc:492] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:12.541231 32544 ts_tablet_manager.cc:1367] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 20:46:12.541630 32544 raft_consensus.cc:348] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.541744 32544 raft_consensus.cc:374] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:12.541824 32544 raft_consensus.cc:729] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:12.541977 32544 consensus_queue.cc:260] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.543660 32545 tablet_bootstrap.cc:492] T 075d3240e24a4532bbb29a068c00c128 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:12.543751 32545 ts_tablet_manager.cc:1367] T 075d3240e24a4532bbb29a068c00c128 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.000s
I20240502 20:46:12.544207 32544 ts_tablet_manager.cc:1398] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:12.544359 32545 raft_consensus.cc:348] T 075d3240e24a4532bbb29a068c00c128 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.544482 32545 raft_consensus.cc:374] T 075d3240e24a4532bbb29a068c00c128 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:12.544574 32545 raft_consensus.cc:729] T 075d3240e24a4532bbb29a068c00c128 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:12.544718 32545 consensus_queue.cc:260] T 075d3240e24a4532bbb29a068c00c128 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.544984 32545 ts_tablet_manager.cc:1398] T 075d3240e24a4532bbb29a068c00c128 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:12.550041 32530 tablet_bootstrap.cc:492] T 4bfb24d76b434cd5b138f868b1f29508 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:12.550308 32530 tablet_bootstrap.cc:492] T 4bfb24d76b434cd5b138f868b1f29508 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap complete.
I20240502 20:46:12.550506 32530 ts_tablet_manager.cc:1367] T 4bfb24d76b434cd5b138f868b1f29508 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.046s user 0.024s sys 0.005s
I20240502 20:46:12.550930 32530 raft_consensus.cc:348] T 4bfb24d76b434cd5b138f868b1f29508 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:12.551079 32530 raft_consensus.cc:729] T 4bfb24d76b434cd5b138f868b1f29508 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:12.551234 32530 consensus_queue.cc:260] T 4bfb24d76b434cd5b138f868b1f29508 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:12.553009 32530 ts_tablet_manager.cc:1398] T 4bfb24d76b434cd5b138f868b1f29508 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:12.553997 29335 tablet_copy_service.cc:342] P 09258ac095664fa99caf0707262cde13: Request end of tablet copy session 315915efcf2b413ca7f9d874338e1b2d-4bfb24d76b434cd5b138f868b1f29508 received from {username='slave'} at 127.22.45.70:37603
I20240502 20:46:12.554105 29335 tablet_copy_service.cc:433] P 09258ac095664fa99caf0707262cde13: ending tablet copy session 315915efcf2b413ca7f9d874338e1b2d-4bfb24d76b434cd5b138f868b1f29508 on tablet 4bfb24d76b434cd5b138f868b1f29508 with peer 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:12.558264 29446 consensus_queue.cc:237] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:12.559669 29182 raft_consensus.cc:1261] T b0e8b1909df34fce92f94d6af5f5c3b6 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.559815 29313 raft_consensus.cc:1261] T b0e8b1909df34fce92f94d6af5f5c3b6 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.560034 31402 consensus_queue.cc:1035] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 20:46:12.560355 29380 consensus_peers.cc:485] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b0e8b1909df34fce92f94d6af5f5c3b6. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:12.560386 31402 consensus_queue.cc:1035] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.562379 32073 raft_consensus.cc:2932] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:12.562835 29313 raft_consensus.cc:2932] T b0e8b1909df34fce92f94d6af5f5c3b6 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:12.564628 29175 raft_consensus.cc:2932] T b0e8b1909df34fce92f94d6af5f5c3b6 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:12.566167 29038 catalog_manager.cc:5484] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:12.583134 29175 consensus_queue.cc:237] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.584483 29446 raft_consensus.cc:1261] T a0e1f5ce5c544045b556cb31e744b169 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.584537 29577 raft_consensus.cc:1261] T a0e1f5ce5c544045b556cb31e744b169 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 20:46:12.584632 29118 consensus_peers.cc:485] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d -> Peer e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): Couldn't send request to peer e6b60803d0764d228c863e2da65e3691. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a0e1f5ce5c544045b556cb31e744b169. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:12.584889 31397 consensus_queue.cc:1035] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.585073 32213 consensus_queue.cc:1035] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.586807 32213 raft_consensus.cc:2932] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:12.587186 29446 raft_consensus.cc:2932] T a0e1f5ce5c544045b556cb31e744b169 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:12.588521 29577 raft_consensus.cc:2932] T a0e1f5ce5c544045b556cb31e744b169 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:12.589712 29042 catalog_manager.cc:5484] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d reported cstate change: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:12.597790 32601 ts_tablet_manager.cc:902] T f19fa5a9a0e840c1bc0bc379a1907383 P e7ab6b9053664ee5aa71c7978e26fe6a: Initiating tablet copy from peer 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:12.598552 32601 tablet_copy_client.cc:308] T f19fa5a9a0e840c1bc0bc379a1907383 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.65:43267
I20240502 20:46:12.602342 29202 tablet_copy_service.cc:140] P 7796ea3fba294d17a34386873969431d: Received BeginTabletCopySession request for tablet f19fa5a9a0e840c1bc0bc379a1907383 from peer e7ab6b9053664ee5aa71c7978e26fe6a ({username='slave'} at 127.22.45.71:50917)
I20240502 20:46:12.602485 29202 tablet_copy_service.cc:161] P 7796ea3fba294d17a34386873969431d: Beginning new tablet copy session on tablet f19fa5a9a0e840c1bc0bc379a1907383 from peer e7ab6b9053664ee5aa71c7978e26fe6a at {username='slave'} at 127.22.45.71:50917: session id = e7ab6b9053664ee5aa71c7978e26fe6a-f19fa5a9a0e840c1bc0bc379a1907383
I20240502 20:46:12.603987 29202 tablet_copy_source_session.cc:205] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:12.606331 32601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f19fa5a9a0e840c1bc0bc379a1907383. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:12.608548 32601 tablet_copy_client.cc:783] T f19fa5a9a0e840c1bc0bc379a1907383 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:12.608717 32601 tablet_copy_client.cc:647] T f19fa5a9a0e840c1bc0bc379a1907383 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:12.608945 29175 consensus_queue.cc:237] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:12.610333 29313 raft_consensus.cc:1261] T f62df6fab4f248e7aae0a57a9ed58575 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 20:46:12.610478 29119 consensus_peers.cc:485] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d -> Peer 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): Couldn't send request to peer 2a9c1af8cf814030bde56b67ca5c2fbe. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f62df6fab4f248e7aae0a57a9ed58575. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:12.610832 29446 raft_consensus.cc:1261] T f62df6fab4f248e7aae0a57a9ed58575 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:12.610901 32590 consensus_queue.cc:1035] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.611315 32590 consensus_queue.cc:1035] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:12.611457 32601 tablet_copy_client.cc:523] T f19fa5a9a0e840c1bc0bc379a1907383 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:12.613102 32601 tablet_bootstrap.cc:492] T f19fa5a9a0e840c1bc0bc379a1907383 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:12.615335 32590 raft_consensus.cc:2932] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:12.615789 29313 raft_consensus.cc:2932] T f62df6fab4f248e7aae0a57a9ed58575 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:12.617923 32615 ts_tablet_manager.cc:902] T 4fd651e80e004dc58d7769f227af58a8 P 2a9c1af8cf814030bde56b67ca5c2fbe: Initiating tablet copy from peer 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:12.618734 32615 tablet_copy_client.cc:308] T 4fd651e80e004dc58d7769f227af58a8 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.65:43267
I20240502 20:46:12.618667 29446 raft_consensus.cc:2932] T f62df6fab4f248e7aae0a57a9ed58575 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:12.618809 29035 catalog_manager.cc:5484] T f62df6fab4f248e7aae0a57a9ed58575 P 09258ac095664fa99caf0707262cde13 reported cstate change: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:12.619697 29202 tablet_copy_service.cc:140] P 7796ea3fba294d17a34386873969431d: Received BeginTabletCopySession request for tablet 4fd651e80e004dc58d7769f227af58a8 from peer 2a9c1af8cf814030bde56b67ca5c2fbe ({username='slave'} at 127.22.45.69:49017)
I20240502 20:46:12.619792 29202 tablet_copy_service.cc:161] P 7796ea3fba294d17a34386873969431d: Beginning new tablet copy session on tablet 4fd651e80e004dc58d7769f227af58a8 from peer 2a9c1af8cf814030bde56b67ca5c2fbe at {username='slave'} at 127.22.45.69:49017: session id = 2a9c1af8cf814030bde56b67ca5c2fbe-4fd651e80e004dc58d7769f227af58a8
I20240502 20:46:12.620647 29202 tablet_copy_source_session.cc:205] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:12.623404 32615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fd651e80e004dc58d7769f227af58a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:12.626194 32615 tablet_copy_client.cc:783] T 4fd651e80e004dc58d7769f227af58a8 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:12.626405 32615 tablet_copy_client.cc:647] T 4fd651e80e004dc58d7769f227af58a8 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:12.628050 32615 tablet_copy_client.cc:523] T 4fd651e80e004dc58d7769f227af58a8 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:12.631229 32615 tablet_bootstrap.cc:492] T 4fd651e80e004dc58d7769f227af58a8 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:12.634948 32111 raft_consensus.cc:482] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:12.635061 32111 raft_consensus.cc:504] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.635998 32111 leader_election.cc:288] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:12.636840 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "075d3240e24a4532bbb29a068c00c128" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:12.636960 29175 raft_consensus.cc:2454] T 075d3240e24a4532bbb29a068c00c128 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:12.637319 31824 leader_election.cc:302] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:12.637501 32111 raft_consensus.cc:2781] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:12.637558 32111 raft_consensus.cc:482] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:12.637598 32111 raft_consensus.cc:3037] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:12.638594 32111 raft_consensus.cc:504] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.638943 32111 leader_election.cc:288] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:12.639344 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "075d3240e24a4532bbb29a068c00c128" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:12.639469 29175 raft_consensus.cc:3037] T 075d3240e24a4532bbb29a068c00c128 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:12.640435 29175 raft_consensus.cc:2454] T 075d3240e24a4532bbb29a068c00c128 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:12.642344 31824 leader_election.cc:302] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:12.642532 32111 raft_consensus.cc:2781] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:12.642622 32111 raft_consensus.cc:686] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Becoming Leader. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Running, Role: LEADER
I20240502 20:46:12.642747 32111 consensus_queue.cc:237] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.643755 32601 tablet_bootstrap.cc:492] T f19fa5a9a0e840c1bc0bc379a1907383 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:12.644029 32601 tablet_bootstrap.cc:492] T f19fa5a9a0e840c1bc0bc379a1907383 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap complete.
I20240502 20:46:12.644196 32601 ts_tablet_manager.cc:1367] T f19fa5a9a0e840c1bc0bc379a1907383 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.031s user 0.024s sys 0.004s
I20240502 20:46:12.644667 32601 raft_consensus.cc:348] T f19fa5a9a0e840c1bc0bc379a1907383 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:12.644800 32601 raft_consensus.cc:729] T f19fa5a9a0e840c1bc0bc379a1907383 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:12.644963 32601 consensus_queue.cc:260] T f19fa5a9a0e840c1bc0bc379a1907383 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:12.645339 32601 ts_tablet_manager.cc:1398] T f19fa5a9a0e840c1bc0bc379a1907383 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 20:46:12.645897 29202 tablet_copy_service.cc:342] P 7796ea3fba294d17a34386873969431d: Request end of tablet copy session e7ab6b9053664ee5aa71c7978e26fe6a-f19fa5a9a0e840c1bc0bc379a1907383 received from {username='slave'} at 127.22.45.71:50917
I20240502 20:46:12.645970 29202 tablet_copy_service.cc:433] P 7796ea3fba294d17a34386873969431d: ending tablet copy session e7ab6b9053664ee5aa71c7978e26fe6a-f19fa5a9a0e840c1bc0bc379a1907383 on tablet f19fa5a9a0e840c1bc0bc379a1907383 with peer e7ab6b9053664ee5aa71c7978e26fe6a
I20240502 20:46:12.647035 29038 catalog_manager.cc:5484] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6b60803d0764d228c863e2da65e3691 (127.22.45.72). New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:12.649137 32601 ts_tablet_manager.cc:902] T c18f9efa9c2043f6b2fad95ad9fd8296 P e7ab6b9053664ee5aa71c7978e26fe6a: Initiating tablet copy from peer dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:12.649524 32601 tablet_copy_client.cc:308] T c18f9efa9c2043f6b2fad95ad9fd8296 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.67:44095
I20240502 20:46:12.650444 29466 tablet_copy_service.cc:140] P dbcdc6ac635b44b185c3eba63f66e9d6: Received BeginTabletCopySession request for tablet c18f9efa9c2043f6b2fad95ad9fd8296 from peer e7ab6b9053664ee5aa71c7978e26fe6a ({username='slave'} at 127.22.45.71:44217)
I20240502 20:46:12.650595 29466 tablet_copy_service.cc:161] P dbcdc6ac635b44b185c3eba63f66e9d6: Beginning new tablet copy session on tablet c18f9efa9c2043f6b2fad95ad9fd8296 from peer e7ab6b9053664ee5aa71c7978e26fe6a at {username='slave'} at 127.22.45.71:44217: session id = e7ab6b9053664ee5aa71c7978e26fe6a-c18f9efa9c2043f6b2fad95ad9fd8296
I20240502 20:46:12.651919 29466 tablet_copy_source_session.cc:205] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:12.653128 32601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c18f9efa9c2043f6b2fad95ad9fd8296. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:12.653115 29038 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:12.655364 32601 tablet_copy_client.cc:783] T c18f9efa9c2043f6b2fad95ad9fd8296 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:12.655506 32601 tablet_copy_client.cc:647] T c18f9efa9c2043f6b2fad95ad9fd8296 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:12.655542 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "075d3240e24a4532bbb29a068c00c128" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:12.655655 31757 raft_consensus.cc:2454] T 075d3240e24a4532bbb29a068c00c128 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:12.655910 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "075d3240e24a4532bbb29a068c00c128" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:12.656013 31756 raft_consensus.cc:3037] T 075d3240e24a4532bbb29a068c00c128 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:12.656893 31756 raft_consensus.cc:2454] T 075d3240e24a4532bbb29a068c00c128 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:12.658041 31494 raft_consensus.cc:1261] T d1546d04e9e844378e6a8ed2f32e3d31 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Refusing update from remote peer 315915efcf2b413ca7f9d874338e1b2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:12.658375 31756 raft_consensus.cc:1261] T d1546d04e9e844378e6a8ed2f32e3d31 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer 315915efcf2b413ca7f9d874338e1b2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:12.658747 32160 consensus_queue.cc:1035] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:12.659148 32160 consensus_queue.cc:1035] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:12.661741 29035 catalog_manager.cc:5576] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887) does not have the latest schema for tablet d1546d04e9e844378e6a8ed2f32e3d31 (table rebalancer_dynamic_table [id=e52fed40cee049f4a63723d5f76fb1a3]). Expected version 1 got 0
I20240502 20:46:12.662515 29038 catalog_manager.cc:5576] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787) does not have the latest schema for tablet d1546d04e9e844378e6a8ed2f32e3d31 (table rebalancer_dynamic_table [id=e52fed40cee049f4a63723d5f76fb1a3]). Expected version 1 got 0
I20240502 20:46:12.663797 32195 tablet.cc:1722] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:12.664130 32195 tablet.cc:1620] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:12.665525 29035 catalog_manager.cc:5576] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887) does not have the latest schema for tablet d1546d04e9e844378e6a8ed2f32e3d31 (table rebalancer_dynamic_table [id=e52fed40cee049f4a63723d5f76fb1a3]). Expected version 1 got 0
I20240502 20:46:12.668610 29035 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=e52fed40cee049f4a63723d5f76fb1a3] alter complete (version 1)
I20240502 20:46:12.671967 32601 tablet_copy_client.cc:523] T c18f9efa9c2043f6b2fad95ad9fd8296 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:12.672359 32641 tablet.cc:1722] T d1546d04e9e844378e6a8ed2f32e3d31 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:12.673120 32641 tablet.cc:1620] T d1546d04e9e844378e6a8ed2f32e3d31 P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:12.673277 32601 tablet_bootstrap.cc:492] T c18f9efa9c2043f6b2fad95ad9fd8296 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:12.673381 32194 tablet.cc:1722] T d1546d04e9e844378e6a8ed2f32e3d31 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:12.673632 32194 tablet.cc:1620] T d1546d04e9e844378e6a8ed2f32e3d31 P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:12.675354 32615 tablet_bootstrap.cc:492] T 4fd651e80e004dc58d7769f227af58a8 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:12.676532 32615 tablet_bootstrap.cc:492] T 4fd651e80e004dc58d7769f227af58a8 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap complete.
I20240502 20:46:12.676828 32615 ts_tablet_manager.cc:1367] T 4fd651e80e004dc58d7769f227af58a8 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.046s user 0.027s sys 0.002s
I20240502 20:46:12.677320 32615 raft_consensus.cc:348] T 4fd651e80e004dc58d7769f227af58a8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:12.677577 32615 raft_consensus.cc:729] T 4fd651e80e004dc58d7769f227af58a8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:12.677909 32615 consensus_queue.cc:260] T 4fd651e80e004dc58d7769f227af58a8 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:12.678323 32615 ts_tablet_manager.cc:1398] T 4fd651e80e004dc58d7769f227af58a8 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:12.679055 29202 tablet_copy_service.cc:342] P 7796ea3fba294d17a34386873969431d: Request end of tablet copy session 2a9c1af8cf814030bde56b67ca5c2fbe-4fd651e80e004dc58d7769f227af58a8 received from {username='slave'} at 127.22.45.69:49017
I20240502 20:46:12.679143 29202 tablet_copy_service.cc:433] P 7796ea3fba294d17a34386873969431d: ending tablet copy session 2a9c1af8cf814030bde56b67ca5c2fbe-4fd651e80e004dc58d7769f227af58a8 on tablet 4fd651e80e004dc58d7769f227af58a8 with peer 2a9c1af8cf814030bde56b67ca5c2fbe
I20240502 20:46:12.686053 32642 ts_tablet_manager.cc:902] T 760d15b2acee4322a26f1f2a965f22cc P e6b60803d0764d228c863e2da65e3691: Initiating tablet copy from peer dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:12.686812 32642 tablet_copy_client.cc:308] T 760d15b2acee4322a26f1f2a965f22cc P e6b60803d0764d228c863e2da65e3691: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.67:44095
I20240502 20:46:12.686957 32530 ts_tablet_manager.cc:902] T 2c6dfe6a92284c5f96094eb16c757222 P 315915efcf2b413ca7f9d874338e1b2d: Initiating tablet copy from peer 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:12.687335 32530 tablet_copy_client.cc:308] T 2c6dfe6a92284c5f96094eb16c757222 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.66:40945
I20240502 20:46:12.688055 29335 tablet_copy_service.cc:140] P 09258ac095664fa99caf0707262cde13: Received BeginTabletCopySession request for tablet 2c6dfe6a92284c5f96094eb16c757222 from peer 315915efcf2b413ca7f9d874338e1b2d ({username='slave'} at 127.22.45.70:37603)
I20240502 20:46:12.688154 29335 tablet_copy_service.cc:161] P 09258ac095664fa99caf0707262cde13: Beginning new tablet copy session on tablet 2c6dfe6a92284c5f96094eb16c757222 from peer 315915efcf2b413ca7f9d874338e1b2d at {username='slave'} at 127.22.45.70:37603: session id = 315915efcf2b413ca7f9d874338e1b2d-2c6dfe6a92284c5f96094eb16c757222
I20240502 20:46:12.689770 29335 tablet_copy_source_session.cc:205] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:12.690529 32530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c6dfe6a92284c5f96094eb16c757222. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:12.692150 29466 tablet_copy_service.cc:140] P dbcdc6ac635b44b185c3eba63f66e9d6: Received BeginTabletCopySession request for tablet 760d15b2acee4322a26f1f2a965f22cc from peer e6b60803d0764d228c863e2da65e3691 ({username='slave'} at 127.22.45.72:44455)
I20240502 20:46:12.692258 29466 tablet_copy_service.cc:161] P dbcdc6ac635b44b185c3eba63f66e9d6: Beginning new tablet copy session on tablet 760d15b2acee4322a26f1f2a965f22cc from peer e6b60803d0764d228c863e2da65e3691 at {username='slave'} at 127.22.45.72:44455: session id = e6b60803d0764d228c863e2da65e3691-760d15b2acee4322a26f1f2a965f22cc
I20240502 20:46:12.692561 32530 tablet_copy_client.cc:783] T 2c6dfe6a92284c5f96094eb16c757222 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:12.692701 32530 tablet_copy_client.cc:647] T 2c6dfe6a92284c5f96094eb16c757222 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:12.693135 29466 tablet_copy_source_session.cc:205] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:12.694347 32530 tablet_copy_client.cc:523] T 2c6dfe6a92284c5f96094eb16c757222 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:12.695081 32642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 760d15b2acee4322a26f1f2a965f22cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:12.695541 32530 tablet_bootstrap.cc:492] T 2c6dfe6a92284c5f96094eb16c757222 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:12.696617 32657 ts_tablet_manager.cc:902] T 94bdf135dd3a47d39e7b76dbfd524386 P e6b60803d0764d228c863e2da65e3691: Initiating tablet copy from peer db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:12.697142 32657 tablet_copy_client.cc:308] T 94bdf135dd3a47d39e7b76dbfd524386 P e6b60803d0764d228c863e2da65e3691: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.68:45897
I20240502 20:46:12.699383 32642 tablet_copy_client.cc:783] T 760d15b2acee4322a26f1f2a965f22cc P e6b60803d0764d228c863e2da65e3691: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:12.699600 32642 tablet_copy_client.cc:647] T 760d15b2acee4322a26f1f2a965f22cc P e6b60803d0764d228c863e2da65e3691: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:12.701589 29597 tablet_copy_service.cc:140] P db17f26a73f3447b92aa9a947e3a7057: Received BeginTabletCopySession request for tablet 94bdf135dd3a47d39e7b76dbfd524386 from peer e6b60803d0764d228c863e2da65e3691 ({username='slave'} at 127.22.45.72:43997)
I20240502 20:46:12.701720 29597 tablet_copy_service.cc:161] P db17f26a73f3447b92aa9a947e3a7057: Beginning new tablet copy session on tablet 94bdf135dd3a47d39e7b76dbfd524386 from peer e6b60803d0764d228c863e2da65e3691 at {username='slave'} at 127.22.45.72:43997: session id = e6b60803d0764d228c863e2da65e3691-94bdf135dd3a47d39e7b76dbfd524386
I20240502 20:46:12.703089 32642 tablet_copy_client.cc:523] T 760d15b2acee4322a26f1f2a965f22cc P e6b60803d0764d228c863e2da65e3691: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:12.703131 29597 tablet_copy_source_session.cc:205] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:12.704278 32657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94bdf135dd3a47d39e7b76dbfd524386. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:12.705549 32642 tablet_bootstrap.cc:492] T 760d15b2acee4322a26f1f2a965f22cc P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:12.708230 32657 tablet_copy_client.cc:783] T 94bdf135dd3a47d39e7b76dbfd524386 P e6b60803d0764d228c863e2da65e3691: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:12.708408 32657 tablet_copy_client.cc:647] T 94bdf135dd3a47d39e7b76dbfd524386 P e6b60803d0764d228c863e2da65e3691: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:12.712114 32657 tablet_copy_client.cc:523] T 94bdf135dd3a47d39e7b76dbfd524386 P e6b60803d0764d228c863e2da65e3691: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:12.713349 32657 tablet_bootstrap.cc:492] T 94bdf135dd3a47d39e7b76dbfd524386 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:12.713565 32601 tablet_bootstrap.cc:492] T c18f9efa9c2043f6b2fad95ad9fd8296 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:12.713820 32601 tablet_bootstrap.cc:492] T c18f9efa9c2043f6b2fad95ad9fd8296 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap complete.
I20240502 20:46:12.714043 32601 ts_tablet_manager.cc:1367] T c18f9efa9c2043f6b2fad95ad9fd8296 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.041s user 0.016s sys 0.005s
I20240502 20:46:12.714677 32601 raft_consensus.cc:348] T c18f9efa9c2043f6b2fad95ad9fd8296 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:12.714802 32601 raft_consensus.cc:729] T c18f9efa9c2043f6b2fad95ad9fd8296 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:12.715934 32601 consensus_queue.cc:260] T c18f9efa9c2043f6b2fad95ad9fd8296 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:12.719616 32601 ts_tablet_manager.cc:1398] T c18f9efa9c2043f6b2fad95ad9fd8296 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20240502 20:46:12.721181 29466 tablet_copy_service.cc:342] P dbcdc6ac635b44b185c3eba63f66e9d6: Request end of tablet copy session e7ab6b9053664ee5aa71c7978e26fe6a-c18f9efa9c2043f6b2fad95ad9fd8296 received from {username='slave'} at 127.22.45.71:44217
I20240502 20:46:12.721279 29466 tablet_copy_service.cc:433] P dbcdc6ac635b44b185c3eba63f66e9d6: ending tablet copy session e7ab6b9053664ee5aa71c7978e26fe6a-c18f9efa9c2043f6b2fad95ad9fd8296 on tablet c18f9efa9c2043f6b2fad95ad9fd8296 with peer e7ab6b9053664ee5aa71c7978e26fe6a
I20240502 20:46:12.724582 31956 raft_consensus.cc:482] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:12.724671 31956 raft_consensus.cc:504] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:12.725090 31956 leader_election.cc:288] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:12.726078 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90bab3d11d424b3abcdc74d5da51cbee" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:12.726197 31756 raft_consensus.cc:2454] T 90bab3d11d424b3abcdc74d5da51cbee P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:12.728667 31559 leader_election.cc:302] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:12.731369 31956 raft_consensus.cc:2781] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:12.731426 31956 raft_consensus.cc:482] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:12.731465 31956 raft_consensus.cc:3037] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:12.732398 31956 raft_consensus.cc:504] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:12.732735 31956 leader_election.cc:288] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Requested vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:12.733170 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90bab3d11d424b3abcdc74d5da51cbee" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:12.733269 31756 raft_consensus.cc:3037] T 90bab3d11d424b3abcdc74d5da51cbee P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:12.734122 31756 raft_consensus.cc:2454] T 90bab3d11d424b3abcdc74d5da51cbee P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:12.734583 31559 leader_election.cc:302] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:12.735423 31956 raft_consensus.cc:2781] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:12.735503 31956 raft_consensus.cc:686] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Becoming Leader. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Running, Role: LEADER
I20240502 20:46:12.735621 31956 consensus_queue.cc:237] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:12.735684 31887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90bab3d11d424b3abcdc74d5da51cbee" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:12.735805 31887 raft_consensus.cc:3037] T 90bab3d11d424b3abcdc74d5da51cbee P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:12.736629 31887 raft_consensus.cc:2454] T 90bab3d11d424b3abcdc74d5da51cbee P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:12.736853 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90bab3d11d424b3abcdc74d5da51cbee" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:12.736941 31888 raft_consensus.cc:2362] T 90bab3d11d424b3abcdc74d5da51cbee P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1. Re-sending same reply.
I20240502 20:46:12.737998 32530 tablet_bootstrap.cc:492] T 2c6dfe6a92284c5f96094eb16c757222 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:12.737916 29035 catalog_manager.cc:5484] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70). New cstate: current_term: 1 leader_uuid: "315915efcf2b413ca7f9d874338e1b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:12.738241 32530 tablet_bootstrap.cc:492] T 2c6dfe6a92284c5f96094eb16c757222 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap complete.
I20240502 20:46:12.738854 32690 ts_tablet_manager.cc:902] T 62f3301937ba4471afcc327be05486cf P e6b60803d0764d228c863e2da65e3691: Initiating tablet copy from peer dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:12.738905 32530 ts_tablet_manager.cc:1367] T 2c6dfe6a92284c5f96094eb16c757222 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.043s user 0.023s sys 0.003s
I20240502 20:46:12.739331 32530 raft_consensus.cc:348] T 2c6dfe6a92284c5f96094eb16c757222 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:12.739434 32530 raft_consensus.cc:729] T 2c6dfe6a92284c5f96094eb16c757222 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:12.739568 32530 consensus_queue.cc:260] T 2c6dfe6a92284c5f96094eb16c757222 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:12.739893 32530 ts_tablet_manager.cc:1398] T 2c6dfe6a92284c5f96094eb16c757222 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:12.740957 29335 tablet_copy_service.cc:342] P 09258ac095664fa99caf0707262cde13: Request end of tablet copy session 315915efcf2b413ca7f9d874338e1b2d-2c6dfe6a92284c5f96094eb16c757222 received from {username='slave'} at 127.22.45.70:37603
I20240502 20:46:12.741027 29335 tablet_copy_service.cc:433] P 09258ac095664fa99caf0707262cde13: ending tablet copy session 315915efcf2b413ca7f9d874338e1b2d-2c6dfe6a92284c5f96094eb16c757222 on tablet 2c6dfe6a92284c5f96094eb16c757222 with peer 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:12.744175 32657 tablet_bootstrap.cc:492] T 94bdf135dd3a47d39e7b76dbfd524386 P e6b60803d0764d228c863e2da65e3691: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:12.744583 32657 tablet_bootstrap.cc:492] T 94bdf135dd3a47d39e7b76dbfd524386 P e6b60803d0764d228c863e2da65e3691: Bootstrap complete.
I20240502 20:46:12.744916 32657 ts_tablet_manager.cc:1367] T 94bdf135dd3a47d39e7b76dbfd524386 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.032s user 0.023s sys 0.005s
I20240502 20:46:12.745297 32657 raft_consensus.cc:348] T 94bdf135dd3a47d39e7b76dbfd524386 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.745388 32657 raft_consensus.cc:729] T 94bdf135dd3a47d39e7b76dbfd524386 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:12.745548 32657 consensus_queue.cc:260] T 94bdf135dd3a47d39e7b76dbfd524386 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.747004 32111 raft_consensus.cc:482] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:12.747087 32111 raft_consensus.cc:504] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.747457 32111 leader_election.cc:288] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:12.747980 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f57649f122e45f9b8ef82ec975e5359" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:12.748097 31494 raft_consensus.cc:2454] T 0f57649f122e45f9b8ef82ec975e5359 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:12.748622 31823 leader_election.cc:302] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:12.748773 32111 raft_consensus.cc:2781] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:12.748819 32111 raft_consensus.cc:482] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:12.748852 32111 raft_consensus.cc:3037] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:12.749716 32111 raft_consensus.cc:504] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.750042 32111 leader_election.cc:288] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:12.751108 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f57649f122e45f9b8ef82ec975e5359" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:12.751232 31494 raft_consensus.cc:3037] T 0f57649f122e45f9b8ef82ec975e5359 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:12.752081 31494 raft_consensus.cc:2454] T 0f57649f122e45f9b8ef82ec975e5359 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:12.752346 31823 leader_election.cc:302] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:12.752666 32111 raft_consensus.cc:2781] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:12.752746 32111 raft_consensus.cc:686] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Becoming Leader. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Running, Role: LEADER
I20240502 20:46:12.752861 32111 consensus_queue.cc:237] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.752905 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f57649f122e45f9b8ef82ec975e5359" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:12.752943 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f57649f122e45f9b8ef82ec975e5359" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:12.753017 31626 raft_consensus.cc:2454] T 0f57649f122e45f9b8ef82ec975e5359 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:12.749930 32657 ts_tablet_manager.cc:1398] T 94bdf135dd3a47d39e7b76dbfd524386 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 20:46:12.754263 29597 tablet_copy_service.cc:342] P db17f26a73f3447b92aa9a947e3a7057: Request end of tablet copy session e6b60803d0764d228c863e2da65e3691-94bdf135dd3a47d39e7b76dbfd524386 received from {username='slave'} at 127.22.45.72:43997
I20240502 20:46:12.754359 29597 tablet_copy_service.cc:433] P db17f26a73f3447b92aa9a947e3a7057: ending tablet copy session e6b60803d0764d228c863e2da65e3691-94bdf135dd3a47d39e7b76dbfd524386 on tablet 94bdf135dd3a47d39e7b76dbfd524386 with peer e6b60803d0764d228c863e2da65e3691
I20240502 20:46:12.759945 32690 tablet_copy_client.cc:308] T 62f3301937ba4471afcc327be05486cf P e6b60803d0764d228c863e2da65e3691: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.67:44095
I20240502 20:46:12.760645 29035 catalog_manager.cc:5484] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6b60803d0764d228c863e2da65e3691 (127.22.45.72). New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:12.763166 32615 ts_tablet_manager.cc:902] T dcbca797b2d14e76b03e379cfe132597 P 2a9c1af8cf814030bde56b67ca5c2fbe: Initiating tablet copy from peer 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:12.763612 32615 tablet_copy_client.cc:308] T dcbca797b2d14e76b03e379cfe132597 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.66:40945
I20240502 20:46:12.764312 32530 ts_tablet_manager.cc:902] T 846f1d6e39e642cfaabf96ad0c52366d P 315915efcf2b413ca7f9d874338e1b2d: Initiating tablet copy from peer dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:12.764686 32530 tablet_copy_client.cc:308] T 846f1d6e39e642cfaabf96ad0c52366d P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.67:44095
I20240502 20:46:12.766316 29466 tablet_copy_service.cc:140] P dbcdc6ac635b44b185c3eba63f66e9d6: Received BeginTabletCopySession request for tablet 62f3301937ba4471afcc327be05486cf from peer e6b60803d0764d228c863e2da65e3691 ({username='slave'} at 127.22.45.72:44455)
I20240502 20:46:12.766430 29466 tablet_copy_service.cc:161] P dbcdc6ac635b44b185c3eba63f66e9d6: Beginning new tablet copy session on tablet 62f3301937ba4471afcc327be05486cf from peer e6b60803d0764d228c863e2da65e3691 at {username='slave'} at 127.22.45.72:44455: session id = e6b60803d0764d228c863e2da65e3691-62f3301937ba4471afcc327be05486cf
I20240502 20:46:12.767654 29466 tablet_copy_source_session.cc:205] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:12.768558 32690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62f3301937ba4471afcc327be05486cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:12.770699 29039 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:12.771010 29466 tablet_copy_service.cc:140] P dbcdc6ac635b44b185c3eba63f66e9d6: Received BeginTabletCopySession request for tablet 846f1d6e39e642cfaabf96ad0c52366d from peer 315915efcf2b413ca7f9d874338e1b2d ({username='slave'} at 127.22.45.70:49775)
I20240502 20:46:12.771111 29466 tablet_copy_service.cc:161] P dbcdc6ac635b44b185c3eba63f66e9d6: Beginning new tablet copy session on tablet 846f1d6e39e642cfaabf96ad0c52366d from peer 315915efcf2b413ca7f9d874338e1b2d at {username='slave'} at 127.22.45.70:49775: session id = 315915efcf2b413ca7f9d874338e1b2d-846f1d6e39e642cfaabf96ad0c52366d
I20240502 20:46:12.771844 29466 tablet_copy_source_session.cc:205] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:12.772650 32530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 846f1d6e39e642cfaabf96ad0c52366d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:12.772920 29335 tablet_copy_service.cc:140] P 09258ac095664fa99caf0707262cde13: Received BeginTabletCopySession request for tablet dcbca797b2d14e76b03e379cfe132597 from peer 2a9c1af8cf814030bde56b67ca5c2fbe ({username='slave'} at 127.22.45.69:52445)
I20240502 20:46:12.773013 29335 tablet_copy_service.cc:161] P 09258ac095664fa99caf0707262cde13: Beginning new tablet copy session on tablet dcbca797b2d14e76b03e379cfe132597 from peer 2a9c1af8cf814030bde56b67ca5c2fbe at {username='slave'} at 127.22.45.69:52445: session id = 2a9c1af8cf814030bde56b67ca5c2fbe-dcbca797b2d14e76b03e379cfe132597
I20240502 20:46:12.773952 29335 tablet_copy_source_session.cc:205] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:12.774703 32530 tablet_copy_client.cc:783] T 846f1d6e39e642cfaabf96ad0c52366d P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:12.774839 32530 tablet_copy_client.cc:647] T 846f1d6e39e642cfaabf96ad0c52366d P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:12.775563 32615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcbca797b2d14e76b03e379cfe132597. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:12.776422 32690 tablet_copy_client.cc:783] T 62f3301937ba4471afcc327be05486cf P e6b60803d0764d228c863e2da65e3691: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:12.776551 32690 tablet_copy_client.cc:647] T 62f3301937ba4471afcc327be05486cf P e6b60803d0764d228c863e2da65e3691: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:12.777681 32615 tablet_copy_client.cc:783] T dcbca797b2d14e76b03e379cfe132597 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:12.777812 32615 tablet_copy_client.cc:647] T dcbca797b2d14e76b03e379cfe132597 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:12.778957 32642 tablet_bootstrap.cc:492] T 760d15b2acee4322a26f1f2a965f22cc P e6b60803d0764d228c863e2da65e3691: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:12.779222 32642 tablet_bootstrap.cc:492] T 760d15b2acee4322a26f1f2a965f22cc P e6b60803d0764d228c863e2da65e3691: Bootstrap complete.
I20240502 20:46:12.779421 32642 ts_tablet_manager.cc:1367] T 760d15b2acee4322a26f1f2a965f22cc P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.074s user 0.025s sys 0.002s
I20240502 20:46:12.779933 32642 raft_consensus.cc:348] T 760d15b2acee4322a26f1f2a965f22cc P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.780073 32642 raft_consensus.cc:729] T 760d15b2acee4322a26f1f2a965f22cc P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:12.780246 32642 consensus_queue.cc:260] T 760d15b2acee4322a26f1f2a965f22cc P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.780016 32690 tablet_copy_client.cc:523] T 62f3301937ba4471afcc327be05486cf P e6b60803d0764d228c863e2da65e3691: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:12.780618 32642 ts_tablet_manager.cc:1398] T 760d15b2acee4322a26f1f2a965f22cc P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:12.780999 29465 tablet_copy_service.cc:342] P dbcdc6ac635b44b185c3eba63f66e9d6: Request end of tablet copy session e6b60803d0764d228c863e2da65e3691-760d15b2acee4322a26f1f2a965f22cc received from {username='slave'} at 127.22.45.72:44455
I20240502 20:46:12.781060 29465 tablet_copy_service.cc:433] P dbcdc6ac635b44b185c3eba63f66e9d6: ending tablet copy session e6b60803d0764d228c863e2da65e3691-760d15b2acee4322a26f1f2a965f22cc on tablet 760d15b2acee4322a26f1f2a965f22cc with peer e6b60803d0764d228c863e2da65e3691
I20240502 20:46:12.781740 32195 tablet.cc:1722] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:12.782033 32195 tablet.cc:1620] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:12.782375 32690 tablet_bootstrap.cc:492] T 62f3301937ba4471afcc327be05486cf P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:12.782925 32530 tablet_copy_client.cc:523] T 846f1d6e39e642cfaabf96ad0c52366d P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:12.783054 32641 tablet.cc:1722] T d1546d04e9e844378e6a8ed2f32e3d31 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:12.783317 32641 tablet.cc:1620] T d1546d04e9e844378e6a8ed2f32e3d31 P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:12.783557 32194 tablet.cc:1722] T d1546d04e9e844378e6a8ed2f32e3d31 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:12.783833 32194 tablet.cc:1620] T d1546d04e9e844378e6a8ed2f32e3d31 P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:12.784061 32530 tablet_bootstrap.cc:492] T 846f1d6e39e642cfaabf96ad0c52366d P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:12.787732 29033 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=e52fed40cee049f4a63723d5f76fb1a3] alter complete (version 2)
I20240502 20:46:12.788985 32615 tablet_copy_client.cc:523] T dcbca797b2d14e76b03e379cfe132597 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:12.790127 32615 tablet_bootstrap.cc:492] T dcbca797b2d14e76b03e379cfe132597 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:12.798079 32601 ts_tablet_manager.cc:902] T 56928692a7284f33bd34e4396925c570 P e7ab6b9053664ee5aa71c7978e26fe6a: Initiating tablet copy from peer 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:12.798557 32601 tablet_copy_client.cc:308] T 56928692a7284f33bd34e4396925c570 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.66:40945
I20240502 20:46:12.803418 29335 tablet_copy_service.cc:140] P 09258ac095664fa99caf0707262cde13: Received BeginTabletCopySession request for tablet 56928692a7284f33bd34e4396925c570 from peer e7ab6b9053664ee5aa71c7978e26fe6a ({username='slave'} at 127.22.45.71:41427)
I20240502 20:46:12.803516 29335 tablet_copy_service.cc:161] P 09258ac095664fa99caf0707262cde13: Beginning new tablet copy session on tablet 56928692a7284f33bd34e4396925c570 from peer e7ab6b9053664ee5aa71c7978e26fe6a at {username='slave'} at 127.22.45.71:41427: session id = e7ab6b9053664ee5aa71c7978e26fe6a-56928692a7284f33bd34e4396925c570
I20240502 20:46:12.804472 29335 tablet_copy_source_session.cc:205] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:12.805368 32601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56928692a7284f33bd34e4396925c570. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:12.810401 32601 tablet_copy_client.cc:783] T 56928692a7284f33bd34e4396925c570 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:12.810570 32601 tablet_copy_client.cc:647] T 56928692a7284f33bd34e4396925c570 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:12.811918 32601 tablet_copy_client.cc:523] T 56928692a7284f33bd34e4396925c570 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:12.813159 32601 tablet_bootstrap.cc:492] T 56928692a7284f33bd34e4396925c570 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:12.822443 32690 tablet_bootstrap.cc:492] T 62f3301937ba4471afcc327be05486cf P e6b60803d0764d228c863e2da65e3691: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:12.822839 32690 tablet_bootstrap.cc:492] T 62f3301937ba4471afcc327be05486cf P e6b60803d0764d228c863e2da65e3691: Bootstrap complete.
I20240502 20:46:12.823063 32690 ts_tablet_manager.cc:1367] T 62f3301937ba4471afcc327be05486cf P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.041s user 0.026s sys 0.001s
I20240502 20:46:12.823841 31602 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:46:12.826660 31475 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:46:12.828202 29295 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:12.831553 29163 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:12.842800 32642 ts_tablet_manager.cc:902] T 616e6b2cacc54145a29a10cc8584bbdf P e6b60803d0764d228c863e2da65e3691: Initiating tablet copy from peer 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:12.843230 32642 tablet_copy_client.cc:308] T 616e6b2cacc54145a29a10cc8584bbdf P e6b60803d0764d228c863e2da65e3691: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.65:43267
I20240502 20:46:12.847745 29202 tablet_copy_service.cc:140] P 7796ea3fba294d17a34386873969431d: Received BeginTabletCopySession request for tablet 616e6b2cacc54145a29a10cc8584bbdf from peer e6b60803d0764d228c863e2da65e3691 ({username='slave'} at 127.22.45.72:55003)
I20240502 20:46:12.847847 29202 tablet_copy_service.cc:161] P 7796ea3fba294d17a34386873969431d: Beginning new tablet copy session on tablet 616e6b2cacc54145a29a10cc8584bbdf from peer e6b60803d0764d228c863e2da65e3691 at {username='slave'} at 127.22.45.72:55003: session id = e6b60803d0764d228c863e2da65e3691-616e6b2cacc54145a29a10cc8584bbdf
I20240502 20:46:12.848537 31868 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:46:12.848821 29202 tablet_copy_source_session.cc:205] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:12.849709 32642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 616e6b2cacc54145a29a10cc8584bbdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:12.851853 32642 tablet_copy_client.cc:783] T 616e6b2cacc54145a29a10cc8584bbdf P e6b60803d0764d228c863e2da65e3691: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:12.851992 32642 tablet_copy_client.cc:647] T 616e6b2cacc54145a29a10cc8584bbdf P e6b60803d0764d228c863e2da65e3691: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:12.854305 31737 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:12.857873 29557 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:12.825831 32690 raft_consensus.cc:348] T 62f3301937ba4471afcc327be05486cf P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.867424 32601 tablet_bootstrap.cc:492] T 56928692a7284f33bd34e4396925c570 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:12.867352 32690 raft_consensus.cc:729] T 62f3301937ba4471afcc327be05486cf P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:12.867698 32601 tablet_bootstrap.cc:492] T 56928692a7284f33bd34e4396925c570 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap complete.
I20240502 20:46:12.867718 32690 consensus_queue.cc:260] T 62f3301937ba4471afcc327be05486cf P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.867893 32601 ts_tablet_manager.cc:1367] T 56928692a7284f33bd34e4396925c570 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.055s user 0.024s sys 0.000s
I20240502 20:46:12.868178 32690 ts_tablet_manager.cc:1398] T 62f3301937ba4471afcc327be05486cf P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.044s user 0.000s sys 0.001s
I20240502 20:46:12.868676 29465 tablet_copy_service.cc:342] P dbcdc6ac635b44b185c3eba63f66e9d6: Request end of tablet copy session e6b60803d0764d228c863e2da65e3691-62f3301937ba4471afcc327be05486cf received from {username='slave'} at 127.22.45.72:44455
I20240502 20:46:12.868754 29465 tablet_copy_service.cc:433] P dbcdc6ac635b44b185c3eba63f66e9d6: ending tablet copy session e6b60803d0764d228c863e2da65e3691-62f3301937ba4471afcc327be05486cf on tablet 62f3301937ba4471afcc327be05486cf with peer e6b60803d0764d228c863e2da65e3691
I20240502 20:46:12.868841 32601 raft_consensus.cc:348] T 56928692a7284f33bd34e4396925c570 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:12.868937 32601 raft_consensus.cc:729] T 56928692a7284f33bd34e4396925c570 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:12.869056 32601 consensus_queue.cc:260] T 56928692a7284f33bd34e4396925c570 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:12.869120 29424 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 20:46:12.869323 32601 ts_tablet_manager.cc:1398] T 56928692a7284f33bd34e4396925c570 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 20:46:12.870225 29335 tablet_copy_service.cc:342] P 09258ac095664fa99caf0707262cde13: Request end of tablet copy session e7ab6b9053664ee5aa71c7978e26fe6a-56928692a7284f33bd34e4396925c570 received from {username='slave'} at 127.22.45.71:41427
I20240502 20:46:12.870304 29335 tablet_copy_service.cc:433] P 09258ac095664fa99caf0707262cde13: ending tablet copy session e7ab6b9053664ee5aa71c7978e26fe6a-56928692a7284f33bd34e4396925c570 on tablet 56928692a7284f33bd34e4396925c570 with peer e7ab6b9053664ee5aa71c7978e26fe6a
I20240502 20:46:12.880816 32642 tablet_copy_client.cc:523] T 616e6b2cacc54145a29a10cc8584bbdf P e6b60803d0764d228c863e2da65e3691: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:12.882299 32642 tablet_bootstrap.cc:492] T 616e6b2cacc54145a29a10cc8584bbdf P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:12.889268 29033 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:12.891402 32731 ts_tablet_manager.cc:902] T d167293496fb4334ae670a62a2f80fa9 P 2a9c1af8cf814030bde56b67ca5c2fbe: Initiating tablet copy from peer 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:12.891999 32731 tablet_copy_client.cc:308] T d167293496fb4334ae670a62a2f80fa9 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.65:43267
I20240502 20:46:12.893568 32530 tablet_bootstrap.cc:492] T 846f1d6e39e642cfaabf96ad0c52366d P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:12.893844 32530 tablet_bootstrap.cc:492] T 846f1d6e39e642cfaabf96ad0c52366d P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap complete.
I20240502 20:46:12.899952 32195 tablet.cc:1722] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:12.900192 32530 ts_tablet_manager.cc:1367] T 846f1d6e39e642cfaabf96ad0c52366d P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.116s user 0.027s sys 0.000s
I20240502 20:46:12.900614 32530 raft_consensus.cc:348] T 846f1d6e39e642cfaabf96ad0c52366d P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:12.900734 32530 raft_consensus.cc:729] T 846f1d6e39e642cfaabf96ad0c52366d P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:12.900887 32530 consensus_queue.cc:260] T 846f1d6e39e642cfaabf96ad0c52366d P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:12.901142 32530 ts_tablet_manager.cc:1398] T 846f1d6e39e642cfaabf96ad0c52366d P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:12.905081 29465 tablet_copy_service.cc:342] P dbcdc6ac635b44b185c3eba63f66e9d6: Request end of tablet copy session 315915efcf2b413ca7f9d874338e1b2d-846f1d6e39e642cfaabf96ad0c52366d received from {username='slave'} at 127.22.45.70:49775
I20240502 20:46:12.905169 29465 tablet_copy_service.cc:433] P dbcdc6ac635b44b185c3eba63f66e9d6: ending tablet copy session 315915efcf2b413ca7f9d874338e1b2d-846f1d6e39e642cfaabf96ad0c52366d on tablet 846f1d6e39e642cfaabf96ad0c52366d with peer 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:12.906908 32194 tablet.cc:1722] T d1546d04e9e844378e6a8ed2f32e3d31 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:12.905439 32641 tablet.cc:1722] T d1546d04e9e844378e6a8ed2f32e3d31 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:12.910142 29202 tablet_copy_service.cc:140] P 7796ea3fba294d17a34386873969431d: Received BeginTabletCopySession request for tablet d167293496fb4334ae670a62a2f80fa9 from peer 2a9c1af8cf814030bde56b67ca5c2fbe ({username='slave'} at 127.22.45.69:49017)
I20240502 20:46:12.910255 29202 tablet_copy_service.cc:161] P 7796ea3fba294d17a34386873969431d: Beginning new tablet copy session on tablet d167293496fb4334ae670a62a2f80fa9 from peer 2a9c1af8cf814030bde56b67ca5c2fbe at {username='slave'} at 127.22.45.69:49017: session id = 2a9c1af8cf814030bde56b67ca5c2fbe-d167293496fb4334ae670a62a2f80fa9
I20240502 20:46:12.911132 29035 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalance_test_table_6" } modify_external_catalogs: true
I20240502 20:46:12.911208 29202 tablet_copy_source_session.cc:205] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:12.911278 29035 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalance_test_table_6" } modify_external_catalogs: true
I20240502 20:46:12.911948 29033 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=e52fed40cee049f4a63723d5f76fb1a3] alter complete (version 3)
I20240502 20:46:12.914042 29035 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 4 replicas of tablet 56928692a7284f33bd34e4396925c570
I20240502 20:46:12.914283 32731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d167293496fb4334ae670a62a2f80fa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:12.914377 29035 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 8ef44f7e7371449882d44860ad8fa9f0
I20240502 20:46:12.914582 29035 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 4 replicas of tablet 4bfb24d76b434cd5b138f868b1f29508
I20240502 20:46:12.914942 32601 ts_tablet_manager.cc:902] T a8916d4762844444bae6fa6f930abbfd P e7ab6b9053664ee5aa71c7978e26fe6a: Initiating tablet copy from peer dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:12.915380 32601 tablet_copy_client.cc:308] T a8916d4762844444bae6fa6f930abbfd P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.67:44095
I20240502 20:46:12.915830 31737 tablet_service.cc:1514] Processing DeleteTablet for tablet 56928692a7284f33bd34e4396925c570 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:12 UTC) from {username='slave'} at 127.0.0.1:33864
I20240502 20:46:12.916249 29465 tablet_copy_service.cc:140] P dbcdc6ac635b44b185c3eba63f66e9d6: Received BeginTabletCopySession request for tablet a8916d4762844444bae6fa6f930abbfd from peer e7ab6b9053664ee5aa71c7978e26fe6a ({username='slave'} at 127.22.45.71:44217)
I20240502 20:46:12.916360 29465 tablet_copy_service.cc:161] P dbcdc6ac635b44b185c3eba63f66e9d6: Beginning new tablet copy session on tablet a8916d4762844444bae6fa6f930abbfd from peer e7ab6b9053664ee5aa71c7978e26fe6a at {username='slave'} at 127.22.45.71:44217: session id = e7ab6b9053664ee5aa71c7978e26fe6a-a8916d4762844444bae6fa6f930abbfd
I20240502 20:46:12.916510 29555 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ef44f7e7371449882d44860ad8fa9f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:12 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:12.916579 29557 tablet_service.cc:1514] Processing DeleteTablet for tablet 56928692a7284f33bd34e4396925c570 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:12 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:12.916731 29556 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bfb24d76b434cd5b138f868b1f29508 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:12 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:12.916759 32088 tablet_replica.cc:330] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:12.916893 32088 raft_consensus.cc:2229] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:12.917068 32088 raft_consensus.cc:2258] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:12.917289 29163 tablet_service.cc:1514] Processing DeleteTablet for tablet 56928692a7284f33bd34e4396925c570 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:12 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:12.917315 29465 tablet_copy_source_session.cc:205] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:12.917459 32089 tablet_replica.cc:330] T 56928692a7284f33bd34e4396925c570 P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:12.917439 29162 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bfb24d76b434cd5b138f868b1f29508 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:12 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:12.917569 32089 raft_consensus.cc:2229] T 56928692a7284f33bd34e4396925c570 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:12.917593 32088 ts_tablet_manager.cc:1875] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:12.917623 29164 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ef44f7e7371449882d44860ad8fa9f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:12 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:12.917744 32089 raft_consensus.cc:2258] T 56928692a7284f33bd34e4396925c570 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:12.918181 29295 tablet_service.cc:1514] Processing DeleteTablet for tablet 56928692a7284f33bd34e4396925c570 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:12 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:12.918200 31602 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bfb24d76b434cd5b138f868b1f29508 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:12 UTC) from {username='slave'} at 127.0.0.1:58346
I20240502 20:46:12.918349 29294 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bfb24d76b434cd5b138f868b1f29508 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:12 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:12.918421 32089 ts_tablet_manager.cc:1875] T 56928692a7284f33bd34e4396925c570 P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:12.918502 32477 tablet_replica.cc:330] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:12.918632 32477 raft_consensus.cc:2229] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:12.918938 32477 raft_consensus.cc:2258] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:12.918959 32601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8916d4762844444bae6fa6f930abbfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:12.919826 32088 ts_tablet_manager.cc:1888] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:12.919899 32088 log.cc:1196] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/8ef44f7e7371449882d44860ad8fa9f0
I20240502 20:46:12.920068 32477 ts_tablet_manager.cc:1875] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:12.920132 32088 ts_tablet_manager.cc:1909] T 8ef44f7e7371449882d44860ad8fa9f0 P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:12.920603 32089 ts_tablet_manager.cc:1888] T 56928692a7284f33bd34e4396925c570 P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:12.920665 32089 log.cc:1196] T 56928692a7284f33bd34e4396925c570 P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/56928692a7284f33bd34e4396925c570
I20240502 20:46:12.920902 32089 ts_tablet_manager.cc:1909] T 56928692a7284f33bd34e4396925c570 P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:12.920935 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet 8ef44f7e7371449882d44860ad8fa9f0 (table rebalance_test_table_6 [id=85861b676eab48d695a81e1716f5dadc]) successfully deleted
I20240502 20:46:12.921139 32088 tablet_replica.cc:330] T 56928692a7284f33bd34e4396925c570 P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:12.921156 32601 tablet_copy_client.cc:783] T a8916d4762844444bae6fa6f930abbfd P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:12.921250 32088 raft_consensus.cc:2229] T 56928692a7284f33bd34e4396925c570 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:12.921293 32601 tablet_copy_client.cc:647] T a8916d4762844444bae6fa6f930abbfd P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:12.921382 32088 raft_consensus.cc:2258] T 56928692a7284f33bd34e4396925c570 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:12.921463 32089 tablet_replica.cc:330] T 4bfb24d76b434cd5b138f868b1f29508 P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:12.921571 32089 raft_consensus.cc:2229] T 4bfb24d76b434cd5b138f868b1f29508 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:12.921655 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 56928692a7284f33bd34e4396925c570 (table rebalance_test_table_6 [id=85861b676eab48d695a81e1716f5dadc]) successfully deleted
I20240502 20:46:12.921808 32089 raft_consensus.cc:2258] T 4bfb24d76b434cd5b138f868b1f29508 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:12.922179 29424 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ef44f7e7371449882d44860ad8fa9f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:12 UTC) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:12.922267 32477 ts_tablet_manager.cc:1888] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:12.922340 32477 log.cc:1196] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/56928692a7284f33bd34e4396925c570
I20240502 20:46:12.922341 32091 tablet_replica.cc:330] T 8ef44f7e7371449882d44860ad8fa9f0 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:12.922405 32088 ts_tablet_manager.cc:1875] T 56928692a7284f33bd34e4396925c570 P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:12.922492 32091 raft_consensus.cc:2229] T 8ef44f7e7371449882d44860ad8fa9f0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:12.922621 32091 raft_consensus.cc:2258] T 8ef44f7e7371449882d44860ad8fa9f0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:12.922924 32477 ts_tablet_manager.cc:1909] T 56928692a7284f33bd34e4396925c570 P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:12.923116 32091 ts_tablet_manager.cc:1875] T 8ef44f7e7371449882d44860ad8fa9f0 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:12.923558 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet 56928692a7284f33bd34e4396925c570 (table rebalance_test_table_6 [id=85861b676eab48d695a81e1716f5dadc]) successfully deleted
I20240502 20:46:12.923754 32477 tablet_replica.cc:330] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:12.923859 32477 raft_consensus.cc:2229] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:12.924041 32477 raft_consensus.cc:2258] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:12.924657 32477 ts_tablet_manager.cc:1875] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:12.925071 32091 ts_tablet_manager.cc:1888] T 8ef44f7e7371449882d44860ad8fa9f0 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:12.925139 32091 log.cc:1196] T 8ef44f7e7371449882d44860ad8fa9f0 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/8ef44f7e7371449882d44860ad8fa9f0
I20240502 20:46:12.925364 32091 ts_tablet_manager.cc:1909] T 8ef44f7e7371449882d44860ad8fa9f0 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting consensus metadata
I20240502 20:46:12.925516 32088 ts_tablet_manager.cc:1888] T 56928692a7284f33bd34e4396925c570 P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:12.925570 32088 log.cc:1196] T 56928692a7284f33bd34e4396925c570 P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/56928692a7284f33bd34e4396925c570
I20240502 20:46:12.925798 32088 ts_tablet_manager.cc:1909] T 56928692a7284f33bd34e4396925c570 P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:12.926110 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet 8ef44f7e7371449882d44860ad8fa9f0 (table rebalance_test_table_6 [id=85861b676eab48d695a81e1716f5dadc]) successfully deleted
I20240502 20:46:12.926498 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet 56928692a7284f33bd34e4396925c570 (table rebalance_test_table_6 [id=85861b676eab48d695a81e1716f5dadc]) successfully deleted
I20240502 20:46:12.926791 32477 ts_tablet_manager.cc:1888] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:12.926916 32477 log.cc:1196] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/4bfb24d76b434cd5b138f868b1f29508
I20240502 20:46:12.926996 32089 ts_tablet_manager.cc:1875] T 4bfb24d76b434cd5b138f868b1f29508 P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:12.927487 32477 ts_tablet_manager.cc:1909] T 4bfb24d76b434cd5b138f868b1f29508 P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:12.927570 32601 tablet_copy_client.cc:523] T a8916d4762844444bae6fa6f930abbfd P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:12.928179 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet 4bfb24d76b434cd5b138f868b1f29508 (table rebalance_test_table_6 [id=85861b676eab48d695a81e1716f5dadc]) successfully deleted
I20240502 20:46:12.928886 32601 tablet_bootstrap.cc:492] T a8916d4762844444bae6fa6f930abbfd P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:12.928927 32089 ts_tablet_manager.cc:1888] T 4bfb24d76b434cd5b138f868b1f29508 P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:12.928983 32089 log.cc:1196] T 4bfb24d76b434cd5b138f868b1f29508 P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/4bfb24d76b434cd5b138f868b1f29508
I20240502 20:46:12.929196 32089 ts_tablet_manager.cc:1909] T 4bfb24d76b434cd5b138f868b1f29508 P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:12.929813 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 4bfb24d76b434cd5b138f868b1f29508 (table rebalance_test_table_6 [id=85861b676eab48d695a81e1716f5dadc]) successfully deleted
I20240502 20:46:12.930032 32089 tablet_replica.cc:330] T 8ef44f7e7371449882d44860ad8fa9f0 P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:12.930183 32089 raft_consensus.cc:2229] T 8ef44f7e7371449882d44860ad8fa9f0 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:12.930330 32089 raft_consensus.cc:2258] T 8ef44f7e7371449882d44860ad8fa9f0 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:12.930539 32615 tablet_bootstrap.cc:492] T dcbca797b2d14e76b03e379cfe132597 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:12.930809 32089 ts_tablet_manager.cc:1875] T 8ef44f7e7371449882d44860ad8fa9f0 P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:12.930985 32615 tablet_bootstrap.cc:492] T dcbca797b2d14e76b03e379cfe132597 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap complete.
I20240502 20:46:12.931211 32615 ts_tablet_manager.cc:1367] T dcbca797b2d14e76b03e379cfe132597 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.141s user 0.024s sys 0.002s
I20240502 20:46:12.931715 32615 raft_consensus.cc:348] T dcbca797b2d14e76b03e379cfe132597 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:12.931897 32615 raft_consensus.cc:729] T dcbca797b2d14e76b03e379cfe132597 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:12.932125 32615 consensus_queue.cc:260] T dcbca797b2d14e76b03e379cfe132597 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:12.932802 32089 ts_tablet_manager.cc:1888] T 8ef44f7e7371449882d44860ad8fa9f0 P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:12.932943 32089 log.cc:1196] T 8ef44f7e7371449882d44860ad8fa9f0 P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/8ef44f7e7371449882d44860ad8fa9f0
I20240502 20:46:12.933221 32089 ts_tablet_manager.cc:1909] T 8ef44f7e7371449882d44860ad8fa9f0 P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:12.933917 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 8ef44f7e7371449882d44860ad8fa9f0 (table rebalance_test_table_6 [id=85861b676eab48d695a81e1716f5dadc]) successfully deleted
I20240502 20:46:12.934166 32088 tablet_replica.cc:330] T 4bfb24d76b434cd5b138f868b1f29508 P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:12.934345 32088 raft_consensus.cc:2229] T 4bfb24d76b434cd5b138f868b1f29508 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:12.934561 32615 ts_tablet_manager.cc:1398] T dcbca797b2d14e76b03e379cfe132597 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240502 20:46:12.934553 32088 raft_consensus.cc:2258] T 4bfb24d76b434cd5b138f868b1f29508 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:12.935317 32088 ts_tablet_manager.cc:1875] T 4bfb24d76b434cd5b138f868b1f29508 P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:12.936484 29335 tablet_copy_service.cc:342] P 09258ac095664fa99caf0707262cde13: Request end of tablet copy session 2a9c1af8cf814030bde56b67ca5c2fbe-dcbca797b2d14e76b03e379cfe132597 received from {username='slave'} at 127.22.45.69:52445
I20240502 20:46:12.936568 29335 tablet_copy_service.cc:433] P 09258ac095664fa99caf0707262cde13: ending tablet copy session 2a9c1af8cf814030bde56b67ca5c2fbe-dcbca797b2d14e76b03e379cfe132597 on tablet dcbca797b2d14e76b03e379cfe132597 with peer 2a9c1af8cf814030bde56b67ca5c2fbe
I20240502 20:46:12.937330 32088 ts_tablet_manager.cc:1888] T 4bfb24d76b434cd5b138f868b1f29508 P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:12.937436 32088 log.cc:1196] T 4bfb24d76b434cd5b138f868b1f29508 P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/4bfb24d76b434cd5b138f868b1f29508
I20240502 20:46:12.937717 32088 ts_tablet_manager.cc:1909] T 4bfb24d76b434cd5b138f868b1f29508 P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:12.937722 32740 tablet_replica.cc:330] T 56928692a7284f33bd34e4396925c570 P e7ab6b9053664ee5aa71c7978e26fe6a: stopping tablet replica
I20240502 20:46:12.937948 32741 tablet_replica.cc:330] T 4bfb24d76b434cd5b138f868b1f29508 P 315915efcf2b413ca7f9d874338e1b2d: stopping tablet replica
I20240502 20:46:12.938114 32741 raft_consensus.cc:2229] T 4bfb24d76b434cd5b138f868b1f29508 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:12.938308 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet 4bfb24d76b434cd5b138f868b1f29508 (table rebalance_test_table_6 [id=85861b676eab48d695a81e1716f5dadc]) successfully deleted
I20240502 20:46:12.938292 32740 raft_consensus.cc:2229] T 56928692a7284f33bd34e4396925c570 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:12.938606 32740 raft_consensus.cc:2258] T 56928692a7284f33bd34e4396925c570 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:12.938711 32741 raft_consensus.cc:2258] T 4bfb24d76b434cd5b138f868b1f29508 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:12.939538 32741 ts_tablet_manager.cc:1875] T 4bfb24d76b434cd5b138f868b1f29508 P 315915efcf2b413ca7f9d874338e1b2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:12.939718 32740 ts_tablet_manager.cc:1875] T 56928692a7284f33bd34e4396925c570 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:12.941578 32740 ts_tablet_manager.cc:1888] T 56928692a7284f33bd34e4396925c570 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:12.941578 32741 ts_tablet_manager.cc:1888] T 4bfb24d76b434cd5b138f868b1f29508 P 315915efcf2b413ca7f9d874338e1b2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:12.941653 32741 log.cc:1196] T 4bfb24d76b434cd5b138f868b1f29508 P 315915efcf2b413ca7f9d874338e1b2d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal/wals/4bfb24d76b434cd5b138f868b1f29508
I20240502 20:46:12.941886 32741 ts_tablet_manager.cc:1909] T 4bfb24d76b434cd5b138f868b1f29508 P 315915efcf2b413ca7f9d874338e1b2d: Deleting consensus metadata
I20240502 20:46:12.942593 29027 catalog_manager.cc:4830] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): tablet 4bfb24d76b434cd5b138f868b1f29508 (table rebalance_test_table_6 [id=85861b676eab48d695a81e1716f5dadc]) successfully deleted
I20240502 20:46:12.942744 32740 log.cc:1196] T 56928692a7284f33bd34e4396925c570 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/wals/56928692a7284f33bd34e4396925c570
I20240502 20:46:12.943042 32740 ts_tablet_manager.cc:1909] T 56928692a7284f33bd34e4396925c570 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting consensus metadata
I20240502 20:46:12.944197 29026 catalog_manager.cc:4830] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): tablet 56928692a7284f33bd34e4396925c570 (table rebalance_test_table_6 [id=85861b676eab48d695a81e1716f5dadc]) successfully deleted
I20240502 20:46:12.950583 32731 tablet_copy_client.cc:783] T d167293496fb4334ae670a62a2f80fa9 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:12.950902 32731 tablet_copy_client.cc:647] T d167293496fb4334ae670a62a2f80fa9 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:12.952013 32731 tablet_copy_client.cc:523] T d167293496fb4334ae670a62a2f80fa9 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:12.953368 32731 tablet_bootstrap.cc:492] T d167293496fb4334ae670a62a2f80fa9 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:12.962145 32601 tablet_bootstrap.cc:492] T a8916d4762844444bae6fa6f930abbfd P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:12.962409 32601 tablet_bootstrap.cc:492] T a8916d4762844444bae6fa6f930abbfd P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap complete.
I20240502 20:46:12.962590 32601 ts_tablet_manager.cc:1367] T a8916d4762844444bae6fa6f930abbfd P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.034s user 0.023s sys 0.000s
I20240502 20:46:12.963104 32601 raft_consensus.cc:348] T a8916d4762844444bae6fa6f930abbfd P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:12.963229 32601 raft_consensus.cc:729] T a8916d4762844444bae6fa6f930abbfd P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:12.963393 32601 consensus_queue.cc:260] T a8916d4762844444bae6fa6f930abbfd P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:12.964233 32601 ts_tablet_manager.cc:1398] T a8916d4762844444bae6fa6f930abbfd P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:12.964661 29465 tablet_copy_service.cc:342] P dbcdc6ac635b44b185c3eba63f66e9d6: Request end of tablet copy session e7ab6b9053664ee5aa71c7978e26fe6a-a8916d4762844444bae6fa6f930abbfd received from {username='slave'} at 127.22.45.71:44217
I20240502 20:46:12.964752 29465 tablet_copy_service.cc:433] P dbcdc6ac635b44b185c3eba63f66e9d6: ending tablet copy session e7ab6b9053664ee5aa71c7978e26fe6a-a8916d4762844444bae6fa6f930abbfd on tablet a8916d4762844444bae6fa6f930abbfd with peer e7ab6b9053664ee5aa71c7978e26fe6a
I20240502 20:46:12.970767 32642 tablet_bootstrap.cc:492] T 616e6b2cacc54145a29a10cc8584bbdf P e6b60803d0764d228c863e2da65e3691: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:12.971117 32642 tablet_bootstrap.cc:492] T 616e6b2cacc54145a29a10cc8584bbdf P e6b60803d0764d228c863e2da65e3691: Bootstrap complete.
I20240502 20:46:12.971318 32642 ts_tablet_manager.cc:1367] T 616e6b2cacc54145a29a10cc8584bbdf P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.089s user 0.024s sys 0.000s
I20240502 20:46:12.971812 32642 raft_consensus.cc:348] T 616e6b2cacc54145a29a10cc8584bbdf P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.971951 32642 raft_consensus.cc:729] T 616e6b2cacc54145a29a10cc8584bbdf P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:12.972112 32642 consensus_queue.cc:260] T 616e6b2cacc54145a29a10cc8584bbdf P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:12.972533 32642 ts_tablet_manager.cc:1398] T 616e6b2cacc54145a29a10cc8584bbdf P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:12.973245 29202 tablet_copy_service.cc:342] P 7796ea3fba294d17a34386873969431d: Request end of tablet copy session e6b60803d0764d228c863e2da65e3691-616e6b2cacc54145a29a10cc8584bbdf received from {username='slave'} at 127.22.45.72:55003
I20240502 20:46:12.973320 29202 tablet_copy_service.cc:433] P 7796ea3fba294d17a34386873969431d: ending tablet copy session e6b60803d0764d228c863e2da65e3691-616e6b2cacc54145a29a10cc8584bbdf on tablet 616e6b2cacc54145a29a10cc8584bbdf with peer e6b60803d0764d228c863e2da65e3691
I20240502 20:46:12.983428 32731 tablet_bootstrap.cc:492] T d167293496fb4334ae670a62a2f80fa9 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:12.983700 32731 tablet_bootstrap.cc:492] T d167293496fb4334ae670a62a2f80fa9 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap complete.
I20240502 20:46:12.983865 32731 ts_tablet_manager.cc:1367] T d167293496fb4334ae670a62a2f80fa9 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.031s user 0.015s sys 0.007s
I20240502 20:46:12.984261 32731 raft_consensus.cc:348] T d167293496fb4334ae670a62a2f80fa9 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:12.984364 32731 raft_consensus.cc:729] T d167293496fb4334ae670a62a2f80fa9 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:12.984495 32731 consensus_queue.cc:260] T d167293496fb4334ae670a62a2f80fa9 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:12.984808 32731 ts_tablet_manager.cc:1398] T d167293496fb4334ae670a62a2f80fa9 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.002s sys 0.001s
I20240502 20:46:12.985311 29202 tablet_copy_service.cc:342] P 7796ea3fba294d17a34386873969431d: Request end of tablet copy session 2a9c1af8cf814030bde56b67ca5c2fbe-d167293496fb4334ae670a62a2f80fa9 received from {username='slave'} at 127.22.45.69:49017
I20240502 20:46:12.985404 29202 tablet_copy_service.cc:433] P 7796ea3fba294d17a34386873969431d: ending tablet copy session 2a9c1af8cf814030bde56b67ca5c2fbe-d167293496fb4334ae670a62a2f80fa9 on tablet d167293496fb4334ae670a62a2f80fa9 with peer 2a9c1af8cf814030bde56b67ca5c2fbe
I20240502 20:46:13.005784 31494 raft_consensus.cc:1203] T 4fd651e80e004dc58d7769f227af58a8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:13.014830 32530 ts_tablet_manager.cc:902] T 2d72ef6e7e8d473baaff8f527c02332b P 315915efcf2b413ca7f9d874338e1b2d: Initiating tablet copy from peer dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:13.015318 32530 tablet_copy_client.cc:308] T 2d72ef6e7e8d473baaff8f527c02332b P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.67:44095
I20240502 20:46:13.015794 29465 tablet_copy_service.cc:140] P dbcdc6ac635b44b185c3eba63f66e9d6: Received BeginTabletCopySession request for tablet 2d72ef6e7e8d473baaff8f527c02332b from peer 315915efcf2b413ca7f9d874338e1b2d ({username='slave'} at 127.22.45.70:49775)
I20240502 20:46:13.015901 29465 tablet_copy_service.cc:161] P dbcdc6ac635b44b185c3eba63f66e9d6: Beginning new tablet copy session on tablet 2d72ef6e7e8d473baaff8f527c02332b from peer 315915efcf2b413ca7f9d874338e1b2d at {username='slave'} at 127.22.45.70:49775: session id = 315915efcf2b413ca7f9d874338e1b2d-2d72ef6e7e8d473baaff8f527c02332b
I20240502 20:46:13.016777 32601 ts_tablet_manager.cc:902] T 9d2cd90615dc4d1894ffe5e80914bdc0 P e7ab6b9053664ee5aa71c7978e26fe6a: Initiating tablet copy from peer dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:13.016942 29465 tablet_copy_source_session.cc:205] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:13.017171 32601 tablet_copy_client.cc:308] T 9d2cd90615dc4d1894ffe5e80914bdc0 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.67:44095
I20240502 20:46:13.017586 32731 ts_tablet_manager.cc:902] T f62df6fab4f248e7aae0a57a9ed58575 P 2a9c1af8cf814030bde56b67ca5c2fbe: Initiating tablet copy from peer 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:13.017747 29040 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:13.017864 29040 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:13.017984 32731 tablet_copy_client.cc:308] T f62df6fab4f248e7aae0a57a9ed58575 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.65:43267
I20240502 20:46:13.018322 29465 tablet_copy_service.cc:140] P dbcdc6ac635b44b185c3eba63f66e9d6: Received BeginTabletCopySession request for tablet 9d2cd90615dc4d1894ffe5e80914bdc0 from peer e7ab6b9053664ee5aa71c7978e26fe6a ({username='slave'} at 127.22.45.71:44217)
I20240502 20:46:13.018410 29465 tablet_copy_service.cc:161] P dbcdc6ac635b44b185c3eba63f66e9d6: Beginning new tablet copy session on tablet 9d2cd90615dc4d1894ffe5e80914bdc0 from peer e7ab6b9053664ee5aa71c7978e26fe6a at {username='slave'} at 127.22.45.71:44217: session id = e7ab6b9053664ee5aa71c7978e26fe6a-9d2cd90615dc4d1894ffe5e80914bdc0
I20240502 20:46:13.019196 29465 tablet_copy_source_session.cc:205] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:13.019388 29202 tablet_copy_service.cc:140] P 7796ea3fba294d17a34386873969431d: Received BeginTabletCopySession request for tablet f62df6fab4f248e7aae0a57a9ed58575 from peer 2a9c1af8cf814030bde56b67ca5c2fbe ({username='slave'} at 127.22.45.69:49017)
I20240502 20:46:13.019490 29202 tablet_copy_service.cc:161] P 7796ea3fba294d17a34386873969431d: Beginning new tablet copy session on tablet f62df6fab4f248e7aae0a57a9ed58575 from peer 2a9c1af8cf814030bde56b67ca5c2fbe at {username='slave'} at 127.22.45.69:49017: session id = 2a9c1af8cf814030bde56b67ca5c2fbe-f62df6fab4f248e7aae0a57a9ed58575
I20240502 20:46:13.020116 32530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d72ef6e7e8d473baaff8f527c02332b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:13.020318 29202 tablet_copy_source_session.cc:205] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:13.021850 32731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f62df6fab4f248e7aae0a57a9ed58575. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:13.022563 32530 tablet_copy_client.cc:783] T 2d72ef6e7e8d473baaff8f527c02332b P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:13.022722 32530 tablet_copy_client.cc:647] T 2d72ef6e7e8d473baaff8f527c02332b P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:13.023773 32530 tablet_copy_client.cc:523] T 2d72ef6e7e8d473baaff8f527c02332b P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:13.023815 32731 tablet_copy_client.cc:783] T f62df6fab4f248e7aae0a57a9ed58575 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:13.023936 32731 tablet_copy_client.cc:647] T f62df6fab4f248e7aae0a57a9ed58575 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:13.024380 32601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d2cd90615dc4d1894ffe5e80914bdc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:13.024966 32530 tablet_bootstrap.cc:492] T 2d72ef6e7e8d473baaff8f527c02332b P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:13.026345 32601 tablet_copy_client.cc:783] T 9d2cd90615dc4d1894ffe5e80914bdc0 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:13.027869 29040 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet d1546d04e9e844378e6a8ed2f32e3d31
I20240502 20:46:13.028719 32601 tablet_copy_client.cc:647] T 9d2cd90615dc4d1894ffe5e80914bdc0 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:13.028668 31475 tablet_service.cc:1514] Processing DeleteTablet for tablet d1546d04e9e844378e6a8ed2f32e3d31 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:13 UTC) from {username='slave'} at 127.0.0.1:40276
I20240502 20:46:13.028867 31737 tablet_service.cc:1514] Processing DeleteTablet for tablet d1546d04e9e844378e6a8ed2f32e3d31 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:13 UTC) from {username='slave'} at 127.0.0.1:33864
I20240502 20:46:13.029012 32740 tablet_replica.cc:330] T d1546d04e9e844378e6a8ed2f32e3d31 P e7ab6b9053664ee5aa71c7978e26fe6a: stopping tablet replica
I20240502 20:46:13.029121 32740 raft_consensus.cc:2229] T d1546d04e9e844378e6a8ed2f32e3d31 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:13.029227 32740 raft_consensus.cc:2258] T d1546d04e9e844378e6a8ed2f32e3d31 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:13.029786 32740 ts_tablet_manager.cc:1875] T d1546d04e9e844378e6a8ed2f32e3d31 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:13.031328 31602 tablet_service.cc:1514] Processing DeleteTablet for tablet d1546d04e9e844378e6a8ed2f32e3d31 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:13 UTC) from {username='slave'} at 127.0.0.1:58346
I20240502 20:46:13.031378 32740 ts_tablet_manager.cc:1888] T d1546d04e9e844378e6a8ed2f32e3d31 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:13.031438 32740 log.cc:1196] T d1546d04e9e844378e6a8ed2f32e3d31 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/wals/d1546d04e9e844378e6a8ed2f32e3d31
I20240502 20:46:13.031445 32731 tablet_copy_client.cc:523] T f62df6fab4f248e7aae0a57a9ed58575 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:13.031484 32741 tablet_replica.cc:330] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d: stopping tablet replica
I20240502 20:46:13.031582 32741 raft_consensus.cc:2229] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:13.031682 32740 ts_tablet_manager.cc:1909] T d1546d04e9e844378e6a8ed2f32e3d31 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting consensus metadata
I20240502 20:46:13.031893 32741 raft_consensus.cc:2258] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:13.032487 32601 tablet_copy_client.cc:523] T 9d2cd90615dc4d1894ffe5e80914bdc0 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:13.032506 29026 catalog_manager.cc:4830] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): tablet d1546d04e9e844378e6a8ed2f32e3d31 (table rebalancer_dynamic_table_new_name [id=e52fed40cee049f4a63723d5f76fb1a3]) successfully deleted
I20240502 20:46:13.032753 32731 tablet_bootstrap.cc:492] T f62df6fab4f248e7aae0a57a9ed58575 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:13.033660 32601 tablet_bootstrap.cc:492] T 9d2cd90615dc4d1894ffe5e80914bdc0 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:13.035393 32741 ts_tablet_manager.cc:1875] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:13.035924 32111 consensus_queue.cc:1035] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:13.037297 32741 ts_tablet_manager.cc:1888] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:13.037359 32741 log.cc:1196] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal/wals/d1546d04e9e844378e6a8ed2f32e3d31
I20240502 20:46:13.037592 32741 ts_tablet_manager.cc:1909] T d1546d04e9e844378e6a8ed2f32e3d31 P 315915efcf2b413ca7f9d874338e1b2d: Deleting consensus metadata
I20240502 20:46:13.040882 32172 consensus_queue.cc:1035] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:13.042943 29027 catalog_manager.cc:4830] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): tablet d1546d04e9e844378e6a8ed2f32e3d31 (table rebalancer_dynamic_table_new_name [id=e52fed40cee049f4a63723d5f76fb1a3]) successfully deleted
I20240502 20:46:13.047147 32615 ts_tablet_manager.cc:902] T e01c5b97e2d14976812691b78a5143c5 P 2a9c1af8cf814030bde56b67ca5c2fbe: Initiating tablet copy from peer 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:13.047655 32615 tablet_copy_client.cc:308] T e01c5b97e2d14976812691b78a5143c5 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.65:43267
I20240502 20:46:13.050562 29202 tablet_copy_service.cc:140] P 7796ea3fba294d17a34386873969431d: Received BeginTabletCopySession request for tablet e01c5b97e2d14976812691b78a5143c5 from peer 2a9c1af8cf814030bde56b67ca5c2fbe ({username='slave'} at 127.22.45.69:49017)
I20240502 20:46:13.050683 29202 tablet_copy_service.cc:161] P 7796ea3fba294d17a34386873969431d: Beginning new tablet copy session on tablet e01c5b97e2d14976812691b78a5143c5 from peer 2a9c1af8cf814030bde56b67ca5c2fbe at {username='slave'} at 127.22.45.69:49017: session id = 2a9c1af8cf814030bde56b67ca5c2fbe-e01c5b97e2d14976812691b78a5143c5
I20240502 20:46:13.051625 29202 tablet_copy_source_session.cc:205] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:13.053478 32615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e01c5b97e2d14976812691b78a5143c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:13.055266 32615 tablet_copy_client.cc:783] T e01c5b97e2d14976812691b78a5143c5 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:13.055404 32615 tablet_copy_client.cc:647] T e01c5b97e2d14976812691b78a5143c5 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:13.061278 32615 tablet_copy_client.cc:523] T e01c5b97e2d14976812691b78a5143c5 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:13.062638 32615 tablet_bootstrap.cc:492] T e01c5b97e2d14976812691b78a5143c5 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:13.066295 32601 tablet_bootstrap.cc:492] T 9d2cd90615dc4d1894ffe5e80914bdc0 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:13.066594 32601 tablet_bootstrap.cc:492] T 9d2cd90615dc4d1894ffe5e80914bdc0 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap complete.
I20240502 20:46:13.066771 32601 ts_tablet_manager.cc:1367] T 9d2cd90615dc4d1894ffe5e80914bdc0 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.033s user 0.019s sys 0.005s
I20240502 20:46:13.067245 32601 raft_consensus.cc:348] T 9d2cd90615dc4d1894ffe5e80914bdc0 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:13.067366 32601 raft_consensus.cc:729] T 9d2cd90615dc4d1894ffe5e80914bdc0 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:13.067502 32601 consensus_queue.cc:260] T 9d2cd90615dc4d1894ffe5e80914bdc0 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:13.067781 32601 ts_tablet_manager.cc:1398] T 9d2cd90615dc4d1894ffe5e80914bdc0 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:13.068403 32731 tablet_bootstrap.cc:492] T f62df6fab4f248e7aae0a57a9ed58575 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:13.068856 32731 tablet_bootstrap.cc:492] T f62df6fab4f248e7aae0a57a9ed58575 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap complete.
I20240502 20:46:13.069048 32731 ts_tablet_manager.cc:1367] T f62df6fab4f248e7aae0a57a9ed58575 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.036s user 0.022s sys 0.000s
I20240502 20:46:13.069186 29465 tablet_copy_service.cc:342] P dbcdc6ac635b44b185c3eba63f66e9d6: Request end of tablet copy session e7ab6b9053664ee5aa71c7978e26fe6a-9d2cd90615dc4d1894ffe5e80914bdc0 received from {username='slave'} at 127.22.45.71:44217
I20240502 20:46:13.069270 29465 tablet_copy_service.cc:433] P dbcdc6ac635b44b185c3eba63f66e9d6: ending tablet copy session e7ab6b9053664ee5aa71c7978e26fe6a-9d2cd90615dc4d1894ffe5e80914bdc0 on tablet 9d2cd90615dc4d1894ffe5e80914bdc0 with peer e7ab6b9053664ee5aa71c7978e26fe6a
I20240502 20:46:13.069928 32731 raft_consensus.cc:348] T f62df6fab4f248e7aae0a57a9ed58575 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.070118 32731 raft_consensus.cc:729] T f62df6fab4f248e7aae0a57a9ed58575 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:13.070580 32731 consensus_queue.cc:260] T f62df6fab4f248e7aae0a57a9ed58575 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.071144 32731 ts_tablet_manager.cc:1398] T f62df6fab4f248e7aae0a57a9ed58575 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:13.071597 29202 tablet_copy_service.cc:342] P 7796ea3fba294d17a34386873969431d: Request end of tablet copy session 2a9c1af8cf814030bde56b67ca5c2fbe-f62df6fab4f248e7aae0a57a9ed58575 received from {username='slave'} at 127.22.45.69:49017
I20240502 20:46:13.071687 29202 tablet_copy_service.cc:433] P 7796ea3fba294d17a34386873969431d: ending tablet copy session 2a9c1af8cf814030bde56b67ca5c2fbe-f62df6fab4f248e7aae0a57a9ed58575 on tablet f62df6fab4f248e7aae0a57a9ed58575 with peer 2a9c1af8cf814030bde56b67ca5c2fbe
I20240502 20:46:13.086845 329 tablet_replica.cc:330] T d1546d04e9e844378e6a8ed2f32e3d31 P 2a9c1af8cf814030bde56b67ca5c2fbe: stopping tablet replica
I20240502 20:46:13.087404 329 raft_consensus.cc:2229] T d1546d04e9e844378e6a8ed2f32e3d31 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:13.094305 29294 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:13.094918 31602 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:46:13.087806 329 raft_consensus.cc:2258] T d1546d04e9e844378e6a8ed2f32e3d31 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:13.092518 32615 tablet_bootstrap.cc:492] T e01c5b97e2d14976812691b78a5143c5 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:13.093080 31475 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:46:13.096585 32615 tablet_bootstrap.cc:492] T e01c5b97e2d14976812691b78a5143c5 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap complete.
I20240502 20:46:13.096808 32615 ts_tablet_manager.cc:1367] T e01c5b97e2d14976812691b78a5143c5 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.034s user 0.023s sys 0.001s
I20240502 20:46:13.097234 32615 raft_consensus.cc:348] T e01c5b97e2d14976812691b78a5143c5 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.098416 32615 raft_consensus.cc:729] T e01c5b97e2d14976812691b78a5143c5 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:13.098624 32615 consensus_queue.cc:260] T e01c5b97e2d14976812691b78a5143c5 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.098896 32615 ts_tablet_manager.cc:1398] T e01c5b97e2d14976812691b78a5143c5 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:13.099366 29202 tablet_copy_service.cc:342] P 7796ea3fba294d17a34386873969431d: Request end of tablet copy session 2a9c1af8cf814030bde56b67ca5c2fbe-e01c5b97e2d14976812691b78a5143c5 received from {username='slave'} at 127.22.45.69:49017
I20240502 20:46:13.099447 29202 tablet_copy_service.cc:433] P 7796ea3fba294d17a34386873969431d: ending tablet copy session 2a9c1af8cf814030bde56b67ca5c2fbe-e01c5b97e2d14976812691b78a5143c5 on tablet e01c5b97e2d14976812691b78a5143c5 with peer 2a9c1af8cf814030bde56b67ca5c2fbe
I20240502 20:46:13.101820 329 ts_tablet_manager.cc:1875] T d1546d04e9e844378e6a8ed2f32e3d31 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:13.107460 29164 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:13.110378 329 ts_tablet_manager.cc:1888] T d1546d04e9e844378e6a8ed2f32e3d31 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:13.110499 329 log.cc:1196] T d1546d04e9e844378e6a8ed2f32e3d31 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/wals/d1546d04e9e844378e6a8ed2f32e3d31
I20240502 20:46:13.115270 329 ts_tablet_manager.cc:1909] T d1546d04e9e844378e6a8ed2f32e3d31 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting consensus metadata
I20240502 20:46:13.118731 29028 catalog_manager.cc:4830] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): tablet d1546d04e9e844378e6a8ed2f32e3d31 (table rebalancer_dynamic_table_new_name [id=e52fed40cee049f4a63723d5f76fb1a3]) successfully deleted
I20240502 20:46:13.120746 31868 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:46:13.121620 29424 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 20:46:13.123332 31737 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:13.125133 32530 tablet_bootstrap.cc:492] T 2d72ef6e7e8d473baaff8f527c02332b P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:13.125491 32530 tablet_bootstrap.cc:492] T 2d72ef6e7e8d473baaff8f527c02332b P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap complete.
I20240502 20:46:13.125705 32530 ts_tablet_manager.cc:1367] T 2d72ef6e7e8d473baaff8f527c02332b P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.101s user 0.023s sys 0.000s
I20240502 20:46:13.126188 32530 raft_consensus.cc:348] T 2d72ef6e7e8d473baaff8f527c02332b P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:13.126401 32530 raft_consensus.cc:729] T 2d72ef6e7e8d473baaff8f527c02332b P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:13.126617 32530 consensus_queue.cc:260] T 2d72ef6e7e8d473baaff8f527c02332b P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:13.127388 32530 ts_tablet_manager.cc:1398] T 2d72ef6e7e8d473baaff8f527c02332b P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:13.130578 29465 tablet_copy_service.cc:342] P dbcdc6ac635b44b185c3eba63f66e9d6: Request end of tablet copy session 315915efcf2b413ca7f9d874338e1b2d-2d72ef6e7e8d473baaff8f527c02332b received from {username='slave'} at 127.22.45.70:49775
I20240502 20:46:13.130669 29465 tablet_copy_service.cc:433] P dbcdc6ac635b44b185c3eba63f66e9d6: ending tablet copy session 315915efcf2b413ca7f9d874338e1b2d-2d72ef6e7e8d473baaff8f527c02332b on tablet 2d72ef6e7e8d473baaff8f527c02332b with peer 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:13.130826 29035 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:13.132133 31888 raft_consensus.cc:1203] T 62f3301937ba4471afcc327be05486cf P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:13.135241 31756 raft_consensus.cc:1203] T c18f9efa9c2043f6b2fad95ad9fd8296 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:13.140878 31626 raft_consensus.cc:1203] T 2c6dfe6a92284c5f96094eb16c757222 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:13.141364 31737 tablet_service.cc:1467] Processing CreateTablet for tablet bc65de5027994a3395f35715d9b871e4 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d82452f64e904290bca28ce7493e0e51]), partition=
I20240502 20:46:13.141600 31602 tablet_service.cc:1467] Processing CreateTablet for tablet bc65de5027994a3395f35715d9b871e4 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d82452f64e904290bca28ce7493e0e51]), partition=
I20240502 20:46:13.141629 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc65de5027994a3395f35715d9b871e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:13.141819 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc65de5027994a3395f35715d9b871e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:13.143114 31868 tablet_service.cc:1467] Processing CreateTablet for tablet bc65de5027994a3395f35715d9b871e4 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d82452f64e904290bca28ce7493e0e51]), partition=
I20240502 20:46:13.143332 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc65de5027994a3395f35715d9b871e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:13.145291 357 tablet_bootstrap.cc:492] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:13.148005 32642 ts_tablet_manager.cc:902] T a0e1f5ce5c544045b556cb31e744b169 P e6b60803d0764d228c863e2da65e3691: Initiating tablet copy from peer 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:13.148428 32642 tablet_copy_client.cc:308] T a0e1f5ce5c544045b556cb31e744b169 P e6b60803d0764d228c863e2da65e3691: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.65:43267
I20240502 20:46:13.148973 29202 tablet_copy_service.cc:140] P 7796ea3fba294d17a34386873969431d: Received BeginTabletCopySession request for tablet a0e1f5ce5c544045b556cb31e744b169 from peer e6b60803d0764d228c863e2da65e3691 ({username='slave'} at 127.22.45.72:55003)
I20240502 20:46:13.149066 29202 tablet_copy_service.cc:161] P 7796ea3fba294d17a34386873969431d: Beginning new tablet copy session on tablet a0e1f5ce5c544045b556cb31e744b169 from peer e6b60803d0764d228c863e2da65e3691 at {username='slave'} at 127.22.45.72:55003: session id = e6b60803d0764d228c863e2da65e3691-a0e1f5ce5c544045b556cb31e744b169
I20240502 20:46:13.149976 29202 tablet_copy_source_session.cc:205] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:13.150806 32642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0e1f5ce5c544045b556cb31e744b169. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:13.152711 32642 tablet_copy_client.cc:783] T a0e1f5ce5c544045b556cb31e744b169 P e6b60803d0764d228c863e2da65e3691: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:13.152691 32160 consensus_queue.cc:1035] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:13.152840 32642 tablet_copy_client.cc:647] T a0e1f5ce5c544045b556cb31e744b169 P e6b60803d0764d228c863e2da65e3691: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:13.154114 357 tablet_bootstrap.cc:654] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:13.154570 356 tablet_bootstrap.cc:492] T bc65de5027994a3395f35715d9b871e4 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:13.155833 356 tablet_bootstrap.cc:654] T bc65de5027994a3395f35715d9b871e4 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:13.155862 32642 tablet_copy_client.cc:523] T a0e1f5ce5c544045b556cb31e744b169 P e6b60803d0764d228c863e2da65e3691: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:13.157091 32642 tablet_bootstrap.cc:492] T a0e1f5ce5c544045b556cb31e744b169 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:13.157210 356 tablet_bootstrap.cc:492] T bc65de5027994a3395f35715d9b871e4 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:13.157333 356 ts_tablet_manager.cc:1367] T bc65de5027994a3395f35715d9b871e4 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:13.157806 356 raft_consensus.cc:348] T bc65de5027994a3395f35715d9b871e4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:13.158002 356 raft_consensus.cc:374] T bc65de5027994a3395f35715d9b871e4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:13.158088 356 raft_consensus.cc:729] T bc65de5027994a3395f35715d9b871e4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:13.158305 356 consensus_queue.cc:260] T bc65de5027994a3395f35715d9b871e4 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:13.158627 29556 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:13.160332 356 ts_tablet_manager.cc:1398] T bc65de5027994a3395f35715d9b871e4 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:13.160641 366 ts_tablet_manager.cc:902] T b0e8b1909df34fce92f94d6af5f5c3b6 P 315915efcf2b413ca7f9d874338e1b2d: Initiating tablet copy from peer dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:13.161229 366 tablet_copy_client.cc:308] T b0e8b1909df34fce92f94d6af5f5c3b6 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.67:44095
I20240502 20:46:13.161758 29465 tablet_copy_service.cc:140] P dbcdc6ac635b44b185c3eba63f66e9d6: Received BeginTabletCopySession request for tablet b0e8b1909df34fce92f94d6af5f5c3b6 from peer 315915efcf2b413ca7f9d874338e1b2d ({username='slave'} at 127.22.45.70:49775)
I20240502 20:46:13.161856 29465 tablet_copy_service.cc:161] P dbcdc6ac635b44b185c3eba63f66e9d6: Beginning new tablet copy session on tablet b0e8b1909df34fce92f94d6af5f5c3b6 from peer 315915efcf2b413ca7f9d874338e1b2d at {username='slave'} at 127.22.45.70:49775: session id = 315915efcf2b413ca7f9d874338e1b2d-b0e8b1909df34fce92f94d6af5f5c3b6
I20240502 20:46:13.162808 29465 tablet_copy_source_session.cc:205] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:13.163709 366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0e8b1909df34fce92f94d6af5f5c3b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:13.166383 366 tablet_copy_client.cc:783] T b0e8b1909df34fce92f94d6af5f5c3b6 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:13.166597 366 tablet_copy_client.cc:647] T b0e8b1909df34fce92f94d6af5f5c3b6 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:13.166692 358 tablet_bootstrap.cc:492] T bc65de5027994a3395f35715d9b871e4 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:13.167595 358 tablet_bootstrap.cc:654] T bc65de5027994a3395f35715d9b871e4 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:13.169229 366 tablet_copy_client.cc:523] T b0e8b1909df34fce92f94d6af5f5c3b6 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:13.170728 366 tablet_bootstrap.cc:492] T b0e8b1909df34fce92f94d6af5f5c3b6 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:13.182873 357 tablet_bootstrap.cc:492] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:13.183008 357 ts_tablet_manager.cc:1367] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.038s user 0.001s sys 0.001s
I20240502 20:46:13.183501 357 raft_consensus.cc:348] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:13.183622 357 raft_consensus.cc:374] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:13.183653 357 raft_consensus.cc:729] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:13.183779 357 consensus_queue.cc:260] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:13.184026 357 ts_tablet_manager.cc:1398] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:13.196156 31757 raft_consensus.cc:1203] T f19fa5a9a0e840c1bc0bc379a1907383 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:13.203076 31956 consensus_queue.cc:1035] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:13.204711 32642 tablet_bootstrap.cc:492] T a0e1f5ce5c544045b556cb31e744b169 P e6b60803d0764d228c863e2da65e3691: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:13.205004 32642 tablet_bootstrap.cc:492] T a0e1f5ce5c544045b556cb31e744b169 P e6b60803d0764d228c863e2da65e3691: Bootstrap complete.
I20240502 20:46:13.205221 32642 ts_tablet_manager.cc:1367] T a0e1f5ce5c544045b556cb31e744b169 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.048s user 0.023s sys 0.000s
I20240502 20:46:13.206559 32642 raft_consensus.cc:348] T a0e1f5ce5c544045b556cb31e744b169 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:13.206688 32642 raft_consensus.cc:729] T a0e1f5ce5c544045b556cb31e744b169 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:13.206835 32642 consensus_queue.cc:260] T a0e1f5ce5c544045b556cb31e744b169 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:13.207139 32642 ts_tablet_manager.cc:1398] T a0e1f5ce5c544045b556cb31e744b169 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:13.207715 29202 tablet_copy_service.cc:342] P 7796ea3fba294d17a34386873969431d: Request end of tablet copy session e6b60803d0764d228c863e2da65e3691-a0e1f5ce5c544045b556cb31e744b169 received from {username='slave'} at 127.22.45.72:55003
I20240502 20:46:13.207813 29202 tablet_copy_service.cc:433] P 7796ea3fba294d17a34386873969431d: ending tablet copy session e6b60803d0764d228c863e2da65e3691-a0e1f5ce5c544045b556cb31e744b169 on tablet a0e1f5ce5c544045b556cb31e744b169 with peer e6b60803d0764d228c863e2da65e3691
I20240502 20:46:13.213522 358 tablet_bootstrap.cc:492] T bc65de5027994a3395f35715d9b871e4 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:13.213634 358 ts_tablet_manager.cc:1367] T bc65de5027994a3395f35715d9b871e4 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.047s user 0.002s sys 0.000s
I20240502 20:46:13.214073 358 raft_consensus.cc:348] T bc65de5027994a3395f35715d9b871e4 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:13.214203 358 raft_consensus.cc:374] T bc65de5027994a3395f35715d9b871e4 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:13.214228 358 raft_consensus.cc:729] T bc65de5027994a3395f35715d9b871e4 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:13.214351 358 consensus_queue.cc:260] T bc65de5027994a3395f35715d9b871e4 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:13.214628 358 ts_tablet_manager.cc:1398] T bc65de5027994a3395f35715d9b871e4 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:13.228737 31888 raft_consensus.cc:1203] T 616e6b2cacc54145a29a10cc8584bbdf P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:13.237241 366 tablet_bootstrap.cc:492] T b0e8b1909df34fce92f94d6af5f5c3b6 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:13.237624 366 tablet_bootstrap.cc:492] T b0e8b1909df34fce92f94d6af5f5c3b6 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap complete.
I20240502 20:46:13.237830 366 ts_tablet_manager.cc:1367] T b0e8b1909df34fce92f94d6af5f5c3b6 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.067s user 0.024s sys 0.000s
I20240502 20:46:13.238369 366 raft_consensus.cc:348] T b0e8b1909df34fce92f94d6af5f5c3b6 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:13.238552 366 raft_consensus.cc:729] T b0e8b1909df34fce92f94d6af5f5c3b6 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:13.238699 366 consensus_queue.cc:260] T b0e8b1909df34fce92f94d6af5f5c3b6 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:13.238977 366 ts_tablet_manager.cc:1398] T b0e8b1909df34fce92f94d6af5f5c3b6 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:13.239589 29465 tablet_copy_service.cc:342] P dbcdc6ac635b44b185c3eba63f66e9d6: Request end of tablet copy session 315915efcf2b413ca7f9d874338e1b2d-b0e8b1909df34fce92f94d6af5f5c3b6 received from {username='slave'} at 127.22.45.70:49775
I20240502 20:46:13.239679 29465 tablet_copy_service.cc:433] P dbcdc6ac635b44b185c3eba63f66e9d6: ending tablet copy session 315915efcf2b413ca7f9d874338e1b2d-b0e8b1909df34fce92f94d6af5f5c3b6 on tablet b0e8b1909df34fce92f94d6af5f5c3b6 with peer 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:13.253999 32172 raft_consensus.cc:482] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:13.254133 32172 raft_consensus.cc:504] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:13.254575 32172 leader_election.cc:288] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:13.254906 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc65de5027994a3395f35715d9b871e4" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:13.255069 31756 raft_consensus.cc:2454] T bc65de5027994a3395f35715d9b871e4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:13.255107 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc65de5027994a3395f35715d9b871e4" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:13.255234 31626 raft_consensus.cc:2454] T bc65de5027994a3395f35715d9b871e4 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:13.255582 31821 leader_election.cc:302] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e6b60803d0764d228c863e2da65e3691, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:13.255884 32172 raft_consensus.cc:2781] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:13.255946 32172 raft_consensus.cc:482] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:13.255985 32172 raft_consensus.cc:3037] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:13.256775 32172 raft_consensus.cc:504] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:13.257083 32172 leader_election.cc:288] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Requested vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:13.257421 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc65de5027994a3395f35715d9b871e4" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:13.257473 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc65de5027994a3395f35715d9b871e4" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:13.257557 31756 raft_consensus.cc:3037] T bc65de5027994a3395f35715d9b871e4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:13.257597 31626 raft_consensus.cc:3037] T bc65de5027994a3395f35715d9b871e4 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:13.258612 31756 raft_consensus.cc:2454] T bc65de5027994a3395f35715d9b871e4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:13.258608 31626 raft_consensus.cc:2454] T bc65de5027994a3395f35715d9b871e4 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:13.258935 31822 leader_election.cc:302] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:13.259167 32172 raft_consensus.cc:2781] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:13.259251 32172 raft_consensus.cc:686] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Becoming Leader. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Running, Role: LEADER
I20240502 20:46:13.259377 32172 consensus_queue.cc:237] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:13.261292 29039 catalog_manager.cc:5484] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6b60803d0764d228c863e2da65e3691 (127.22.45.72). New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:13.284792 31888 raft_consensus.cc:1203] T 760d15b2acee4322a26f1f2a965f22cc P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:13.295589 31888 raft_consensus.cc:1203] T 94bdf135dd3a47d39e7b76dbfd524386 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:13.300726 31626 raft_consensus.cc:1203] T 846f1d6e39e642cfaabf96ad0c52366d P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:13.302897 32111 consensus_queue.cc:1035] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:13.307024 31626 raft_consensus.cc:3037] T 0f57649f122e45f9b8ef82ec975e5359 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:13.308912 32111 consensus_queue.cc:1035] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:13.312789 29294 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:13.315737 29037 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:13.322772 31602 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:46:13.323873 29556 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:13.326864 31475 tablet_service.cc:1467] Processing CreateTablet for tablet 28d6a4d9c92141f794d6ee04335cf9b8 (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=57203d81e8ab41bcb24361b9b6107024]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:13.327067 31736 tablet_service.cc:1467] Processing CreateTablet for tablet 63a41aabb95d4e8a9071c7673b257c47 (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=57203d81e8ab41bcb24361b9b6107024]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:13.327186 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28d6a4d9c92141f794d6ee04335cf9b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:13.327304 31736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63a41aabb95d4e8a9071c7673b257c47. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:13.329295 31737 tablet_service.cc:1467] Processing CreateTablet for tablet 28d6a4d9c92141f794d6ee04335cf9b8 (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=57203d81e8ab41bcb24361b9b6107024]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:13.329360 356 tablet_bootstrap.cc:492] T 63a41aabb95d4e8a9071c7673b257c47 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:13.329506 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28d6a4d9c92141f794d6ee04335cf9b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:13.330278 356 tablet_bootstrap.cc:654] T 63a41aabb95d4e8a9071c7673b257c47 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:13.331748 29294 tablet_service.cc:1467] Processing CreateTablet for tablet 63a41aabb95d4e8a9071c7673b257c47 (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=57203d81e8ab41bcb24361b9b6107024]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:13.332018 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63a41aabb95d4e8a9071c7673b257c47. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:13.333827 429 tablet_bootstrap.cc:492] T 63a41aabb95d4e8a9071c7673b257c47 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:13.334772 31602 tablet_service.cc:1467] Processing CreateTablet for tablet b718cecdc692408bbeea91bc9fe21fcb (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=57203d81e8ab41bcb24361b9b6107024]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:13.334926 29164 tablet_service.cc:1467] Processing CreateTablet for tablet b718cecdc692408bbeea91bc9fe21fcb (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=57203d81e8ab41bcb24361b9b6107024]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:13.335017 429 tablet_bootstrap.cc:654] T 63a41aabb95d4e8a9071c7673b257c47 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:13.335029 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b718cecdc692408bbeea91bc9fe21fcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:13.335215 29164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b718cecdc692408bbeea91bc9fe21fcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:13.336885 431 tablet_bootstrap.cc:492] T b718cecdc692408bbeea91bc9fe21fcb P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:13.336977 358 tablet_bootstrap.cc:492] T b718cecdc692408bbeea91bc9fe21fcb P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:13.337352 31737 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:13.337874 358 tablet_bootstrap.cc:654] T b718cecdc692408bbeea91bc9fe21fcb P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:13.337888 431 tablet_bootstrap.cc:654] T b718cecdc692408bbeea91bc9fe21fcb P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:13.339397 431 tablet_bootstrap.cc:492] T b718cecdc692408bbeea91bc9fe21fcb P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:13.339349 29556 tablet_service.cc:1467] Processing CreateTablet for tablet 28d6a4d9c92141f794d6ee04335cf9b8 (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=57203d81e8ab41bcb24361b9b6107024]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:13.339489 431 ts_tablet_manager.cc:1367] T b718cecdc692408bbeea91bc9fe21fcb P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:13.339608 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28d6a4d9c92141f794d6ee04335cf9b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:13.339910 431 raft_consensus.cc:348] T b718cecdc692408bbeea91bc9fe21fcb P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.340008 431 raft_consensus.cc:374] T b718cecdc692408bbeea91bc9fe21fcb P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:13.340034 431 raft_consensus.cc:729] T b718cecdc692408bbeea91bc9fe21fcb P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:13.340126 431 consensus_queue.cc:260] T b718cecdc692408bbeea91bc9fe21fcb P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.340394 431 ts_tablet_manager.cc:1398] T b718cecdc692408bbeea91bc9fe21fcb P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:13.342149 31465 tablet_service.cc:1467] Processing CreateTablet for tablet b718cecdc692408bbeea91bc9fe21fcb (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=57203d81e8ab41bcb24361b9b6107024]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:13.342407 31465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b718cecdc692408bbeea91bc9fe21fcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:13.344492 436 tablet_bootstrap.cc:492] T b718cecdc692408bbeea91bc9fe21fcb P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:13.344578 31474 tablet_service.cc:1467] Processing CreateTablet for tablet 63a41aabb95d4e8a9071c7673b257c47 (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=57203d81e8ab41bcb24361b9b6107024]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:13.344815 31474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63a41aabb95d4e8a9071c7673b257c47. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:13.345659 436 tablet_bootstrap.cc:654] T b718cecdc692408bbeea91bc9fe21fcb P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:13.349473 436 tablet_bootstrap.cc:492] T b718cecdc692408bbeea91bc9fe21fcb P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:13.349582 436 ts_tablet_manager.cc:1367] T b718cecdc692408bbeea91bc9fe21fcb P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 20:46:13.349985 436 raft_consensus.cc:348] T b718cecdc692408bbeea91bc9fe21fcb P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.350081 436 raft_consensus.cc:374] T b718cecdc692408bbeea91bc9fe21fcb P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:13.350106 436 raft_consensus.cc:729] T b718cecdc692408bbeea91bc9fe21fcb P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:13.350221 436 consensus_queue.cc:260] T b718cecdc692408bbeea91bc9fe21fcb P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.350493 436 ts_tablet_manager.cc:1398] T b718cecdc692408bbeea91bc9fe21fcb P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:13.350647 436 tablet_bootstrap.cc:492] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:13.351277 435 tablet_bootstrap.cc:492] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:13.351454 436 tablet_bootstrap.cc:654] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:13.351984 31474 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:46:13.352129 435 tablet_bootstrap.cc:654] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:13.354629 358 tablet_bootstrap.cc:492] T b718cecdc692408bbeea91bc9fe21fcb P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:13.354737 358 ts_tablet_manager.cc:1367] T b718cecdc692408bbeea91bc9fe21fcb P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.001s
I20240502 20:46:13.354920 29164 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:13.355217 358 raft_consensus.cc:348] T b718cecdc692408bbeea91bc9fe21fcb P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.355343 358 raft_consensus.cc:374] T b718cecdc692408bbeea91bc9fe21fcb P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:13.355374 358 raft_consensus.cc:729] T b718cecdc692408bbeea91bc9fe21fcb P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:13.355510 358 consensus_queue.cc:260] T b718cecdc692408bbeea91bc9fe21fcb P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.355829 358 ts_tablet_manager.cc:1398] T b718cecdc692408bbeea91bc9fe21fcb P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:13.358709 29424 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 20:46:13.363941 436 tablet_bootstrap.cc:492] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:13.364045 436 ts_tablet_manager.cc:1367] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 20:46:13.364440 436 raft_consensus.cc:348] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:13.364923 436 raft_consensus.cc:374] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:13.364981 436 raft_consensus.cc:729] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:13.365142 436 consensus_queue.cc:260] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:13.365553 436 ts_tablet_manager.cc:1398] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:13.365728 436 tablet_bootstrap.cc:492] T 28d6a4d9c92141f794d6ee04335cf9b8 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:13.367306 436 tablet_bootstrap.cc:654] T 28d6a4d9c92141f794d6ee04335cf9b8 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:13.368974 29037 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:13.373924 31626 raft_consensus.cc:1261] T bc65de5027994a3395f35715d9b871e4 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:13.374413 32172 consensus_queue.cc:1035] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:13.375244 31756 raft_consensus.cc:1261] T bc65de5027994a3395f35715d9b871e4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:13.375739 32172 consensus_queue.cc:1035] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:13.375975 31494 raft_consensus.cc:1203] T dcbca797b2d14e76b03e379cfe132597 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:13.377601 29037 catalog_manager.cc:5576] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751) does not have the latest schema for tablet bc65de5027994a3395f35715d9b871e4 (table rebalancer_dynamic_table [id=d82452f64e904290bca28ce7493e0e51]). Expected version 1 got 0
I20240502 20:46:13.379532 29037 catalog_manager.cc:5576] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787) does not have the latest schema for tablet bc65de5027994a3395f35715d9b871e4 (table rebalancer_dynamic_table [id=d82452f64e904290bca28ce7493e0e51]). Expected version 1 got 0
I20240502 20:46:13.381706 429 tablet_bootstrap.cc:492] T 63a41aabb95d4e8a9071c7673b257c47 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:13.381807 429 ts_tablet_manager.cc:1367] T 63a41aabb95d4e8a9071c7673b257c47 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.048s user 0.001s sys 0.000s
I20240502 20:46:13.381956 356 tablet_bootstrap.cc:492] T 63a41aabb95d4e8a9071c7673b257c47 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:13.382056 356 ts_tablet_manager.cc:1367] T 63a41aabb95d4e8a9071c7673b257c47 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.053s user 0.000s sys 0.002s
I20240502 20:46:13.382236 429 raft_consensus.cc:348] T 63a41aabb95d4e8a9071c7673b257c47 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:13.382339 429 raft_consensus.cc:374] T 63a41aabb95d4e8a9071c7673b257c47 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:13.382369 429 raft_consensus.cc:729] T 63a41aabb95d4e8a9071c7673b257c47 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:13.382438 356 raft_consensus.cc:348] T 63a41aabb95d4e8a9071c7673b257c47 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:13.382561 356 raft_consensus.cc:374] T 63a41aabb95d4e8a9071c7673b257c47 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:13.382516 429 consensus_queue.cc:260] T 63a41aabb95d4e8a9071c7673b257c47 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:13.382592 356 raft_consensus.cc:729] T 63a41aabb95d4e8a9071c7673b257c47 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:13.382721 356 consensus_queue.cc:260] T 63a41aabb95d4e8a9071c7673b257c47 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:13.382788 429 ts_tablet_manager.cc:1398] T 63a41aabb95d4e8a9071c7673b257c47 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:13.383001 356 ts_tablet_manager.cc:1398] T 63a41aabb95d4e8a9071c7673b257c47 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:13.383188 356 tablet_bootstrap.cc:492] T 28d6a4d9c92141f794d6ee04335cf9b8 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:13.383993 356 tablet_bootstrap.cc:654] T 28d6a4d9c92141f794d6ee04335cf9b8 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:13.388288 334 mvcc.cc:205] Tried to move back new op lower bound from 7023340639734255616 to 7023340639271931904. Current Snapshot: MvccSnapshot[applied={T|T < 7023340639734255616}]
I20240502 20:46:13.388273 32195 tablet.cc:1722] T bc65de5027994a3395f35715d9b871e4 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:13.388629 32195 tablet.cc:1620] T bc65de5027994a3395f35715d9b871e4 P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:13.391081 356 tablet_bootstrap.cc:492] T 28d6a4d9c92141f794d6ee04335cf9b8 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:13.393795 29037 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=d82452f64e904290bca28ce7493e0e51] alter complete (version 1)
I20240502 20:46:13.395138 356 ts_tablet_manager.cc:1367] T 28d6a4d9c92141f794d6ee04335cf9b8 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.002s
I20240502 20:46:13.395244 31868 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:13.395767 356 raft_consensus.cc:348] T 28d6a4d9c92141f794d6ee04335cf9b8 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.395962 356 raft_consensus.cc:374] T 28d6a4d9c92141f794d6ee04335cf9b8 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:13.396045 356 raft_consensus.cc:729] T 28d6a4d9c92141f794d6ee04335cf9b8 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:13.396248 356 consensus_queue.cc:260] T 28d6a4d9c92141f794d6ee04335cf9b8 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.396622 356 ts_tablet_manager.cc:1398] T 28d6a4d9c92141f794d6ee04335cf9b8 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:13.404690 32194 tablet.cc:1722] T bc65de5027994a3395f35715d9b871e4 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:13.405035 32194 tablet.cc:1620] T bc65de5027994a3395f35715d9b871e4 P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:13.407240 31494 raft_consensus.cc:1203] T d167293496fb4334ae670a62a2f80fa9 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:13.416603 435 tablet_bootstrap.cc:492] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:13.416829 435 ts_tablet_manager.cc:1367] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.066s user 0.002s sys 0.000s
I20240502 20:46:13.417527 435 raft_consensus.cc:348] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.417822 435 raft_consensus.cc:374] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:13.417901 435 raft_consensus.cc:729] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:13.418118 435 consensus_queue.cc:260] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.418272 29040 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalance_test_table_8" } modify_external_catalogs: true
I20240502 20:46:13.418393 29040 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalance_test_table_8" } modify_external_catalogs: true
I20240502 20:46:13.419390 435 ts_tablet_manager.cc:1398] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:13.420926 29040 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 4 replicas of tablet f62df6fab4f248e7aae0a57a9ed58575
I20240502 20:46:13.421281 29040 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet a76bead2ebc94234818f070ae4608d0f
I20240502 20:46:13.421448 29294 tablet_service.cc:1514] Processing DeleteTablet for tablet f62df6fab4f248e7aae0a57a9ed58575 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:13 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:13.421579 29040 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 89bea12cd67e4406a59d691037f385b6
I20240502 20:46:13.421641 32477 tablet_replica.cc:330] T f62df6fab4f248e7aae0a57a9ed58575 P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:13.421805 32477 raft_consensus.cc:2229] T f62df6fab4f248e7aae0a57a9ed58575 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:13.421988 32477 raft_consensus.cc:2258] T f62df6fab4f248e7aae0a57a9ed58575 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:13.423403 29164 tablet_service.cc:1514] Processing DeleteTablet for tablet f62df6fab4f248e7aae0a57a9ed58575 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:13 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:13.423559 29162 tablet_service.cc:1514] Processing DeleteTablet for tablet a76bead2ebc94234818f070ae4608d0f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:13 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:13.423408 29163 tablet_service.cc:1514] Processing DeleteTablet for tablet 89bea12cd67e4406a59d691037f385b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:13 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:13.424226 32477 ts_tablet_manager.cc:1875] T f62df6fab4f248e7aae0a57a9ed58575 P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:13.426630 32089 tablet_replica.cc:330] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:13.426805 32089 raft_consensus.cc:2229] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:13.427078 29556 tablet_service.cc:1514] Processing DeleteTablet for tablet a76bead2ebc94234818f070ae4608d0f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:13 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:13.427121 32089 raft_consensus.cc:2258] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:13.427249 29557 tablet_service.cc:1514] Processing DeleteTablet for tablet 89bea12cd67e4406a59d691037f385b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:13 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:13.427426 32088 tablet_replica.cc:330] T a76bead2ebc94234818f070ae4608d0f P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:13.427553 32088 raft_consensus.cc:2229] T a76bead2ebc94234818f070ae4608d0f P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:13.427685 32088 raft_consensus.cc:2258] T a76bead2ebc94234818f070ae4608d0f P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:13.427973 31474 tablet_service.cc:1514] Processing DeleteTablet for tablet f62df6fab4f248e7aae0a57a9ed58575 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:13 UTC) from {username='slave'} at 127.0.0.1:40276
I20240502 20:46:13.427979 32089 ts_tablet_manager.cc:1875] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:13.428181 329 tablet_replica.cc:330] T f62df6fab4f248e7aae0a57a9ed58575 P 2a9c1af8cf814030bde56b67ca5c2fbe: stopping tablet replica
I20240502 20:46:13.428189 32088 ts_tablet_manager.cc:1875] T a76bead2ebc94234818f070ae4608d0f P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:13.428319 329 raft_consensus.cc:2229] T f62df6fab4f248e7aae0a57a9ed58575 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:13.428426 329 raft_consensus.cc:2258] T f62df6fab4f248e7aae0a57a9ed58575 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:13.428645 29294 tablet_service.cc:1514] Processing DeleteTablet for tablet a76bead2ebc94234818f070ae4608d0f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:13 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:13.428835 32477 ts_tablet_manager.cc:1888] T f62df6fab4f248e7aae0a57a9ed58575 P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:13.428893 32477 log.cc:1196] T f62df6fab4f248e7aae0a57a9ed58575 P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/f62df6fab4f248e7aae0a57a9ed58575
I20240502 20:46:13.429147 32477 ts_tablet_manager.cc:1909] T f62df6fab4f248e7aae0a57a9ed58575 P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:13.429252 329 ts_tablet_manager.cc:1875] T f62df6fab4f248e7aae0a57a9ed58575 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:13.429944 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet f62df6fab4f248e7aae0a57a9ed58575 (table rebalance_test_table_8 [id=59a4763b0fad4ad084f27fcea88d8624]) successfully deleted
I20240502 20:46:13.430186 32477 tablet_replica.cc:330] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:13.430239 32089 ts_tablet_manager.cc:1888] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:13.430313 32089 log.cc:1196] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/f62df6fab4f248e7aae0a57a9ed58575
I20240502 20:46:13.430305 32477 raft_consensus.cc:2229] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:13.430320 32088 ts_tablet_manager.cc:1888] T a76bead2ebc94234818f070ae4608d0f P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:13.430378 32088 log.cc:1196] T a76bead2ebc94234818f070ae4608d0f P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/a76bead2ebc94234818f070ae4608d0f
I20240502 20:46:13.430522 32477 raft_consensus.cc:2258] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:13.430727 29424 tablet_service.cc:1514] Processing DeleteTablet for tablet f62df6fab4f248e7aae0a57a9ed58575 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:13 UTC) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:13.430976 32477 ts_tablet_manager.cc:1875] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:13.431011 29426 tablet_service.cc:1514] Processing DeleteTablet for tablet 89bea12cd67e4406a59d691037f385b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:13 UTC) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:13.431694 329 ts_tablet_manager.cc:1888] T f62df6fab4f248e7aae0a57a9ed58575 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:13.431764 329 log.cc:1196] T f62df6fab4f248e7aae0a57a9ed58575 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/wals/f62df6fab4f248e7aae0a57a9ed58575
I20240502 20:46:13.431885 32089 ts_tablet_manager.cc:1909] T f62df6fab4f248e7aae0a57a9ed58575 P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:13.432019 329 ts_tablet_manager.cc:1909] T f62df6fab4f248e7aae0a57a9ed58575 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting consensus metadata
I20240502 20:46:13.432545 32089 tablet_replica.cc:330] T a76bead2ebc94234818f070ae4608d0f P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:13.432672 32089 raft_consensus.cc:2229] T a76bead2ebc94234818f070ae4608d0f P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:13.432796 32089 raft_consensus.cc:2258] T a76bead2ebc94234818f070ae4608d0f P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:13.433257 32089 ts_tablet_manager.cc:1875] T a76bead2ebc94234818f070ae4608d0f P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:13.433427 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet f62df6fab4f248e7aae0a57a9ed58575 (table rebalance_test_table_8 [id=59a4763b0fad4ad084f27fcea88d8624]) successfully deleted
I20240502 20:46:13.433605 29028 catalog_manager.cc:4830] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): tablet f62df6fab4f248e7aae0a57a9ed58575 (table rebalance_test_table_8 [id=59a4763b0fad4ad084f27fcea88d8624]) successfully deleted
I20240502 20:46:13.435122 32088 ts_tablet_manager.cc:1909] T a76bead2ebc94234818f070ae4608d0f P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:13.435266 32089 ts_tablet_manager.cc:1888] T a76bead2ebc94234818f070ae4608d0f P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:13.435325 32089 log.cc:1196] T a76bead2ebc94234818f070ae4608d0f P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/a76bead2ebc94234818f070ae4608d0f
I20240502 20:46:13.435539 32089 ts_tablet_manager.cc:1909] T a76bead2ebc94234818f070ae4608d0f P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:13.435922 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet a76bead2ebc94234818f070ae4608d0f (table rebalance_test_table_8 [id=59a4763b0fad4ad084f27fcea88d8624]) successfully deleted
I20240502 20:46:13.435983 32088 tablet_replica.cc:330] T 89bea12cd67e4406a59d691037f385b6 P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:13.436089 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet a76bead2ebc94234818f070ae4608d0f (table rebalance_test_table_8 [id=59a4763b0fad4ad084f27fcea88d8624]) successfully deleted
I20240502 20:46:13.436262 32477 ts_tablet_manager.cc:1888] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:13.436273 32088 raft_consensus.cc:2229] T 89bea12cd67e4406a59d691037f385b6 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:13.436318 32089 tablet_replica.cc:330] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:13.436383 32477 log.cc:1196] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/a76bead2ebc94234818f070ae4608d0f
I20240502 20:46:13.436434 32088 raft_consensus.cc:2258] T 89bea12cd67e4406a59d691037f385b6 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:13.436499 32089 raft_consensus.cc:2229] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:13.436671 32089 raft_consensus.cc:2258] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:13.436851 32477 ts_tablet_manager.cc:1909] T a76bead2ebc94234818f070ae4608d0f P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:13.437028 32088 ts_tablet_manager.cc:1875] T 89bea12cd67e4406a59d691037f385b6 P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:13.437129 32089 ts_tablet_manager.cc:1875] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:13.438067 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet a76bead2ebc94234818f070ae4608d0f (table rebalance_test_table_8 [id=59a4763b0fad4ad084f27fcea88d8624]) successfully deleted
I20240502 20:46:13.438987 32089 ts_tablet_manager.cc:1888] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:13.439067 32089 log.cc:1196] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/89bea12cd67e4406a59d691037f385b6
I20240502 20:46:13.439088 32088 ts_tablet_manager.cc:1888] T 89bea12cd67e4406a59d691037f385b6 P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:13.439190 32088 log.cc:1196] T 89bea12cd67e4406a59d691037f385b6 P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/89bea12cd67e4406a59d691037f385b6
I20240502 20:46:13.439306 32089 ts_tablet_manager.cc:1909] T 89bea12cd67e4406a59d691037f385b6 P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:13.439476 32088 ts_tablet_manager.cc:1909] T 89bea12cd67e4406a59d691037f385b6 P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:13.440024 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet 89bea12cd67e4406a59d691037f385b6 (table rebalance_test_table_8 [id=59a4763b0fad4ad084f27fcea88d8624]) successfully deleted
I20240502 20:46:13.442432 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 89bea12cd67e4406a59d691037f385b6 (table rebalance_test_table_8 [id=59a4763b0fad4ad084f27fcea88d8624]) successfully deleted
I20240502 20:46:13.442761 445 tablet_replica.cc:330] T f62df6fab4f248e7aae0a57a9ed58575 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:13.442915 445 raft_consensus.cc:2229] T f62df6fab4f248e7aae0a57a9ed58575 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:13.443084 445 raft_consensus.cc:2258] T f62df6fab4f248e7aae0a57a9ed58575 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:13.443686 445 ts_tablet_manager.cc:1875] T f62df6fab4f248e7aae0a57a9ed58575 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:13.445803 445 ts_tablet_manager.cc:1888] T f62df6fab4f248e7aae0a57a9ed58575 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:13.445876 445 log.cc:1196] T f62df6fab4f248e7aae0a57a9ed58575 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/f62df6fab4f248e7aae0a57a9ed58575
I20240502 20:46:13.446161 445 ts_tablet_manager.cc:1909] T f62df6fab4f248e7aae0a57a9ed58575 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting consensus metadata
I20240502 20:46:13.446946 445 tablet_replica.cc:330] T 89bea12cd67e4406a59d691037f385b6 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:13.447016 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet f62df6fab4f248e7aae0a57a9ed58575 (table rebalance_test_table_8 [id=59a4763b0fad4ad084f27fcea88d8624]) successfully deleted
I20240502 20:46:13.447072 445 raft_consensus.cc:2229] T 89bea12cd67e4406a59d691037f385b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:13.447188 445 raft_consensus.cc:2258] T 89bea12cd67e4406a59d691037f385b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:13.447661 445 ts_tablet_manager.cc:1875] T 89bea12cd67e4406a59d691037f385b6 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:13.448688 444 tablet.cc:1722] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:13.449455 445 ts_tablet_manager.cc:1888] T 89bea12cd67e4406a59d691037f385b6 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:13.449575 445 log.cc:1196] T 89bea12cd67e4406a59d691037f385b6 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/89bea12cd67e4406a59d691037f385b6
I20240502 20:46:13.449721 444 tablet.cc:1620] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:13.449855 445 ts_tablet_manager.cc:1909] T 89bea12cd67e4406a59d691037f385b6 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting consensus metadata
I20240502 20:46:13.450750 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet 89bea12cd67e4406a59d691037f385b6 (table rebalance_test_table_8 [id=59a4763b0fad4ad084f27fcea88d8624]) successfully deleted
I20240502 20:46:13.451285 436 tablet_bootstrap.cc:492] T 28d6a4d9c92141f794d6ee04335cf9b8 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:13.451376 436 ts_tablet_manager.cc:1367] T 28d6a4d9c92141f794d6ee04335cf9b8 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.086s user 0.002s sys 0.000s
I20240502 20:46:13.451893 436 raft_consensus.cc:348] T 28d6a4d9c92141f794d6ee04335cf9b8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.452004 436 raft_consensus.cc:374] T 28d6a4d9c92141f794d6ee04335cf9b8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:13.452034 436 raft_consensus.cc:729] T 28d6a4d9c92141f794d6ee04335cf9b8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:13.452153 436 consensus_queue.cc:260] T 28d6a4d9c92141f794d6ee04335cf9b8 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.452448 436 ts_tablet_manager.cc:1398] T 28d6a4d9c92141f794d6ee04335cf9b8 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:13.480746 31756 raft_consensus.cc:1203] T 9d2cd90615dc4d1894ffe5e80914bdc0 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:13.496495 29037 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:13.502784 32108 raft_consensus.cc:482] T b718cecdc692408bbeea91bc9fe21fcb P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:13.502916 32108 raft_consensus.cc:504] T b718cecdc692408bbeea91bc9fe21fcb P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.503433 32108 leader_election.cc:288] T b718cecdc692408bbeea91bc9fe21fcb P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:13.503952 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b718cecdc692408bbeea91bc9fe21fcb" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:13.504032 444 tablet.cc:1722] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:13.504093 29175 raft_consensus.cc:2454] T b718cecdc692408bbeea91bc9fe21fcb P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:13.504390 444 tablet.cc:1620] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:13.504382 31431 leader_election.cc:302] T b718cecdc692408bbeea91bc9fe21fcb P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:13.504544 32108 raft_consensus.cc:2781] T b718cecdc692408bbeea91bc9fe21fcb P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:13.504588 32108 raft_consensus.cc:482] T b718cecdc692408bbeea91bc9fe21fcb P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:13.504621 32108 raft_consensus.cc:3037] T b718cecdc692408bbeea91bc9fe21fcb P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:13.505470 32194 tablet.cc:1722] T bc65de5027994a3395f35715d9b871e4 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:13.505662 32108 raft_consensus.cc:504] T b718cecdc692408bbeea91bc9fe21fcb P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.505775 32194 tablet.cc:1620] T bc65de5027994a3395f35715d9b871e4 P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:13.506002 32108 leader_election.cc:288] T b718cecdc692408bbeea91bc9fe21fcb P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:13.508761 32195 tablet.cc:1722] T bc65de5027994a3395f35715d9b871e4 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:13.509066 32195 tablet.cc:1620] T bc65de5027994a3395f35715d9b871e4 P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:13.510594 29035 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=d82452f64e904290bca28ce7493e0e51] alter complete (version 2)
I20240502 20:46:13.513267 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b718cecdc692408bbeea91bc9fe21fcb" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:13.513402 29175 raft_consensus.cc:3037] T b718cecdc692408bbeea91bc9fe21fcb P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:13.514096 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b718cecdc692408bbeea91bc9fe21fcb" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:13.514165 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b718cecdc692408bbeea91bc9fe21fcb" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:13.514232 31626 raft_consensus.cc:2454] T b718cecdc692408bbeea91bc9fe21fcb P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:13.514330 29175 raft_consensus.cc:2454] T b718cecdc692408bbeea91bc9fe21fcb P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:13.514672 31431 leader_election.cc:302] T b718cecdc692408bbeea91bc9fe21fcb P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:13.515000 32108 raft_consensus.cc:2781] T b718cecdc692408bbeea91bc9fe21fcb P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:13.515156 32108 raft_consensus.cc:686] T b718cecdc692408bbeea91bc9fe21fcb P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Becoming Leader. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Running, Role: LEADER
I20240502 20:46:13.515302 32108 consensus_queue.cc:237] T b718cecdc692408bbeea91bc9fe21fcb P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.516794 29035 catalog_manager.cc:5484] T b718cecdc692408bbeea91bc9fe21fcb P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69). New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:13.523146 31756 raft_consensus.cc:1203] T a8916d4762844444bae6fa6f930abbfd P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:13.530066 31626 raft_consensus.cc:1203] T 2d72ef6e7e8d473baaff8f527c02332b P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:13.533257 32174 raft_consensus.cc:482] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:13.533394 32174 raft_consensus.cc:504] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.534022 32174 leader_election.cc:288] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:13.540931 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d6a4d9c92141f794d6ee04335cf9b8" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:13.541105 31494 raft_consensus.cc:2454] T 28d6a4d9c92141f794d6ee04335cf9b8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:13.541388 29512 leader_election.cc:302] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:13.541554 32174 raft_consensus.cc:2781] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:13.541597 32174 raft_consensus.cc:482] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:13.541628 32174 raft_consensus.cc:3037] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:13.542773 32174 raft_consensus.cc:504] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.543156 32174 leader_election.cc:288] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Requested vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:13.544040 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d6a4d9c92141f794d6ee04335cf9b8" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:13.544159 31494 raft_consensus.cc:3037] T 28d6a4d9c92141f794d6ee04335cf9b8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:13.545111 31494 raft_consensus.cc:2454] T 28d6a4d9c92141f794d6ee04335cf9b8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:13.545693 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d6a4d9c92141f794d6ee04335cf9b8" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:13.545742 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d6a4d9c92141f794d6ee04335cf9b8" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:13.545822 31756 raft_consensus.cc:2454] T 28d6a4d9c92141f794d6ee04335cf9b8 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:13.546066 29512 leader_election.cc:302] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:13.546227 32174 raft_consensus.cc:2781] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:13.546317 32174 raft_consensus.cc:686] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Becoming Leader. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Running, Role: LEADER
I20240502 20:46:13.546500 32174 consensus_queue.cc:237] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.548185 29041 catalog_manager.cc:5484] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: term changed from 0 to 1, leader changed from <none> to db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:13.563124 31888 raft_consensus.cc:1203] T a0e1f5ce5c544045b556cb31e744b169 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:13.590732 31494 raft_consensus.cc:1203] T e01c5b97e2d14976812691b78a5143c5 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:13.614883 29036 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:13.620198 444 tablet.cc:1722] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:13.621373 32194 tablet.cc:1722] T bc65de5027994a3395f35715d9b871e4 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:13.621735 32195 tablet.cc:1722] T bc65de5027994a3395f35715d9b871e4 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:13.624635 32108 raft_consensus.cc:482] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:13.624756 32108 raft_consensus.cc:504] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:13.625294 32108 leader_election.cc:288] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:13.625706 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63a41aabb95d4e8a9071c7673b257c47" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:13.625834 29314 raft_consensus.cc:2454] T 63a41aabb95d4e8a9071c7673b257c47 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:13.626148 31430 leader_election.cc:302] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 2a9c1af8cf814030bde56b67ca5c2fbe; no voters:
I20240502 20:46:13.626348 32108 raft_consensus.cc:2781] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:13.626397 32108 raft_consensus.cc:482] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:13.626437 32108 raft_consensus.cc:3037] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:13.627342 32108 raft_consensus.cc:504] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:13.627705 32108 leader_election.cc:288] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Requested vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:13.628312 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63a41aabb95d4e8a9071c7673b257c47" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:13.628422 29314 raft_consensus.cc:3037] T 63a41aabb95d4e8a9071c7673b257c47 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:13.629278 29314 raft_consensus.cc:2454] T 63a41aabb95d4e8a9071c7673b257c47 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:13.629565 31430 leader_election.cc:302] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 2a9c1af8cf814030bde56b67ca5c2fbe; no voters:
I20240502 20:46:13.629782 29036 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=d82452f64e904290bca28ce7493e0e51] alter complete (version 3)
I20240502 20:46:13.630353 32108 raft_consensus.cc:2781] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:13.630439 32108 raft_consensus.cc:686] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Becoming Leader. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Running, Role: LEADER
I20240502 20:46:13.630587 32108 consensus_queue.cc:237] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:13.630736 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63a41aabb95d4e8a9071c7673b257c47" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:13.630849 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63a41aabb95d4e8a9071c7673b257c47" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:13.630848 31757 raft_consensus.cc:3037] T 63a41aabb95d4e8a9071c7673b257c47 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:13.631906 31757 raft_consensus.cc:2454] T 63a41aabb95d4e8a9071c7673b257c47 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:13.632663 29036 catalog_manager.cc:5484] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69). New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:13.662312 29557 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:13.663537 31602 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:46:13.666255 31737 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:13.666584 29294 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:13.673179 31474 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:46:13.674065 31626 raft_consensus.cc:1203] T b0e8b1909df34fce92f94d6af5f5c3b6 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:13.680251 29426 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 20:46:13.689112 29163 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:13.706023 31868 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:13.731040 29036 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:13.731209 29036 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:13.734031 29036 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet bc65de5027994a3395f35715d9b871e4
I20240502 20:46:13.735476 31868 tablet_service.cc:1514] Processing DeleteTablet for tablet bc65de5027994a3395f35715d9b871e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:13 UTC) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:13.735947 507 tablet_replica.cc:330] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:13.736050 31602 tablet_service.cc:1514] Processing DeleteTablet for tablet bc65de5027994a3395f35715d9b871e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:13 UTC) from {username='slave'} at 127.0.0.1:58346
I20240502 20:46:13.736279 507 raft_consensus.cc:2229] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:13.736343 31737 tablet_service.cc:1514] Processing DeleteTablet for tablet bc65de5027994a3395f35715d9b871e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:13 UTC) from {username='slave'} at 127.0.0.1:33864
I20240502 20:46:13.736624 507 raft_consensus.cc:2258] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:13.737929 507 ts_tablet_manager.cc:1875] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:13.738422 508 tablet_replica.cc:330] T bc65de5027994a3395f35715d9b871e4 P 315915efcf2b413ca7f9d874338e1b2d: stopping tablet replica
I20240502 20:46:13.738675 508 raft_consensus.cc:2229] T bc65de5027994a3395f35715d9b871e4 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:13.738873 508 raft_consensus.cc:2258] T bc65de5027994a3395f35715d9b871e4 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:13.739540 509 tablet_replica.cc:330] T bc65de5027994a3395f35715d9b871e4 P e7ab6b9053664ee5aa71c7978e26fe6a: stopping tablet replica
I20240502 20:46:13.739722 509 raft_consensus.cc:2229] T bc65de5027994a3395f35715d9b871e4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:13.739890 509 raft_consensus.cc:2258] T bc65de5027994a3395f35715d9b871e4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:13.739938 508 ts_tablet_manager.cc:1875] T bc65de5027994a3395f35715d9b871e4 P 315915efcf2b413ca7f9d874338e1b2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:13.740587 507 ts_tablet_manager.cc:1888] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:13.740671 507 log.cc:1196] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/bc65de5027994a3395f35715d9b871e4
I20240502 20:46:13.740847 509 ts_tablet_manager.cc:1875] T bc65de5027994a3395f35715d9b871e4 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:13.740952 507 ts_tablet_manager.cc:1909] T bc65de5027994a3395f35715d9b871e4 P e6b60803d0764d228c863e2da65e3691: Deleting consensus metadata
I20240502 20:46:13.741943 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet bc65de5027994a3395f35715d9b871e4 (table rebalancer_dynamic_table_new_name [id=d82452f64e904290bca28ce7493e0e51]) successfully deleted
I20240502 20:46:13.742136 508 ts_tablet_manager.cc:1888] T bc65de5027994a3395f35715d9b871e4 P 315915efcf2b413ca7f9d874338e1b2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:13.742223 508 log.cc:1196] T bc65de5027994a3395f35715d9b871e4 P 315915efcf2b413ca7f9d874338e1b2d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal/wals/bc65de5027994a3395f35715d9b871e4
I20240502 20:46:13.742545 508 ts_tablet_manager.cc:1909] T bc65de5027994a3395f35715d9b871e4 P 315915efcf2b413ca7f9d874338e1b2d: Deleting consensus metadata
I20240502 20:46:13.743018 509 ts_tablet_manager.cc:1888] T bc65de5027994a3395f35715d9b871e4 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:13.743113 509 log.cc:1196] T bc65de5027994a3395f35715d9b871e4 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/wals/bc65de5027994a3395f35715d9b871e4
I20240502 20:46:13.743319 29027 catalog_manager.cc:4830] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): tablet bc65de5027994a3395f35715d9b871e4 (table rebalancer_dynamic_table_new_name [id=d82452f64e904290bca28ce7493e0e51]) successfully deleted
I20240502 20:46:13.743522 509 ts_tablet_manager.cc:1909] T bc65de5027994a3395f35715d9b871e4 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting consensus metadata
I20240502 20:46:13.745188 29026 catalog_manager.cc:4830] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): tablet bc65de5027994a3395f35715d9b871e4 (table rebalancer_dynamic_table_new_name [id=d82452f64e904290bca28ce7493e0e51]) successfully deleted
I20240502 20:46:13.818096 29163 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:13.819727 31602 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:46:13.822896 31474 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:46:13.823369 29294 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:13.827332 29426 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 20:46:13.827404 31868 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:46:13.836813 31737 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:13.837760 29035 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:13.846431 29557 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:13.856655 31474 tablet_service.cc:1467] Processing CreateTablet for tablet 2a03d782933b4c8bb95a97c142b22388 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d409c40457214940a8c89c7071ba7749]), partition=
I20240502 20:46:13.856930 31474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a03d782933b4c8bb95a97c142b22388. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:13.858132 31602 tablet_service.cc:1467] Processing CreateTablet for tablet 2a03d782933b4c8bb95a97c142b22388 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d409c40457214940a8c89c7071ba7749]), partition=
I20240502 20:46:13.858359 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a03d782933b4c8bb95a97c142b22388. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:13.858991 436 tablet_bootstrap.cc:492] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:13.859397 31737 tablet_service.cc:1467] Processing CreateTablet for tablet 2a03d782933b4c8bb95a97c142b22388 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d409c40457214940a8c89c7071ba7749]), partition=
I20240502 20:46:13.859611 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a03d782933b4c8bb95a97c142b22388. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:13.859845 436 tablet_bootstrap.cc:654] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:13.861356 356 tablet_bootstrap.cc:492] T 2a03d782933b4c8bb95a97c142b22388 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:13.862175 356 tablet_bootstrap.cc:654] T 2a03d782933b4c8bb95a97c142b22388 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:13.868450 436 tablet_bootstrap.cc:492] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:13.868552 436 ts_tablet_manager.cc:1367] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 20:46:13.868985 436 raft_consensus.cc:348] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.869089 436 raft_consensus.cc:374] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:13.869115 436 raft_consensus.cc:729] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:13.869235 436 consensus_queue.cc:260] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.869488 436 ts_tablet_manager.cc:1398] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:13.880447 561 tablet_bootstrap.cc:492] T 2a03d782933b4c8bb95a97c142b22388 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:13.881428 561 tablet_bootstrap.cc:654] T 2a03d782933b4c8bb95a97c142b22388 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:13.882777 561 tablet_bootstrap.cc:492] T 2a03d782933b4c8bb95a97c142b22388 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:13.882872 561 ts_tablet_manager.cc:1367] T 2a03d782933b4c8bb95a97c142b22388 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:13.883304 561 raft_consensus.cc:348] T 2a03d782933b4c8bb95a97c142b22388 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.883407 561 raft_consensus.cc:374] T 2a03d782933b4c8bb95a97c142b22388 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:13.883436 561 raft_consensus.cc:729] T 2a03d782933b4c8bb95a97c142b22388 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:13.883569 561 consensus_queue.cc:260] T 2a03d782933b4c8bb95a97c142b22388 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.883834 561 ts_tablet_manager.cc:1398] T 2a03d782933b4c8bb95a97c142b22388 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:13.903594 356 tablet_bootstrap.cc:492] T 2a03d782933b4c8bb95a97c142b22388 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:13.903718 356 ts_tablet_manager.cc:1367] T 2a03d782933b4c8bb95a97c142b22388 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.042s user 0.000s sys 0.002s
I20240502 20:46:13.904139 356 raft_consensus.cc:348] T 2a03d782933b4c8bb95a97c142b22388 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.905756 356 raft_consensus.cc:374] T 2a03d782933b4c8bb95a97c142b22388 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:13.905804 356 raft_consensus.cc:729] T 2a03d782933b4c8bb95a97c142b22388 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:13.905928 356 consensus_queue.cc:260] T 2a03d782933b4c8bb95a97c142b22388 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.906181 356 ts_tablet_manager.cc:1398] T 2a03d782933b4c8bb95a97c142b22388 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:13.927690 29035 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalance_test_table_10" } modify_external_catalogs: true
I20240502 20:46:13.927843 29035 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalance_test_table_10" } modify_external_catalogs: true
I20240502 20:46:13.930279 29035 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 4 replicas of tablet 2c6dfe6a92284c5f96094eb16c757222
I20240502 20:46:13.930711 29035 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 88a8cf3a902d4e5bbbaf47a5202d514a
I20240502 20:46:13.930974 29035 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 9b609e0004524a67bb4eea7b3c5d58fb
I20240502 20:46:13.930922 29446 consensus_queue.cc:237] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:13.931249 29162 tablet_service.cc:1514] Processing DeleteTablet for tablet 88a8cf3a902d4e5bbbaf47a5202d514a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:13 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:13.931432 32089 tablet_replica.cc:330] T 88a8cf3a902d4e5bbbaf47a5202d514a P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:13.931520 29163 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c6dfe6a92284c5f96094eb16c757222 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:13 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:13.931577 32089 raft_consensus.cc:2229] T 88a8cf3a902d4e5bbbaf47a5202d514a P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:13.931671 29162 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b609e0004524a67bb4eea7b3c5d58fb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:13 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:13.931720 32089 raft_consensus.cc:2258] T 88a8cf3a902d4e5bbbaf47a5202d514a P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:13.931704 29557 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c6dfe6a92284c5f96094eb16c757222 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:13 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:13.931883 29294 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c6dfe6a92284c5f96094eb16c757222 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:13 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:13.931960 29295 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b609e0004524a67bb4eea7b3c5d58fb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:13 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:13.932113 32477 tablet_replica.cc:330] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:13.932163 32088 tablet_replica.cc:330] T 2c6dfe6a92284c5f96094eb16c757222 P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:13.932251 32477 raft_consensus.cc:2229] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:13.932250 32089 ts_tablet_manager.cc:1875] T 88a8cf3a902d4e5bbbaf47a5202d514a P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:13.932288 32088 raft_consensus.cc:2229] T 2c6dfe6a92284c5f96094eb16c757222 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:13.932463 32088 raft_consensus.cc:2258] T 2c6dfe6a92284c5f96094eb16c757222 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:13.932479 32477 raft_consensus.cc:2258] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:13.932957 32477 ts_tablet_manager.cc:1875] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:13.932957 32088 ts_tablet_manager.cc:1875] T 2c6dfe6a92284c5f96094eb16c757222 P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:13.934078 32089 ts_tablet_manager.cc:1888] T 88a8cf3a902d4e5bbbaf47a5202d514a P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:13.934140 32089 log.cc:1196] T 88a8cf3a902d4e5bbbaf47a5202d514a P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/88a8cf3a902d4e5bbbaf47a5202d514a
I20240502 20:46:13.934370 32089 ts_tablet_manager.cc:1909] T 88a8cf3a902d4e5bbbaf47a5202d514a P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:13.934998 32089 tablet_replica.cc:330] T 2c6dfe6a92284c5f96094eb16c757222 P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:13.935117 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 88a8cf3a902d4e5bbbaf47a5202d514a (table rebalance_test_table_10 [id=79c2e912ffdb4aae9cf78414231a842e]) successfully deleted
I20240502 20:46:13.935144 32477 ts_tablet_manager.cc:1888] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:13.935146 32088 ts_tablet_manager.cc:1888] T 2c6dfe6a92284c5f96094eb16c757222 P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:13.935210 32088 log.cc:1196] T 2c6dfe6a92284c5f96094eb16c757222 P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/2c6dfe6a92284c5f96094eb16c757222
I20240502 20:46:13.935218 32477 log.cc:1196] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/2c6dfe6a92284c5f96094eb16c757222
I20240502 20:46:13.935277 32089 raft_consensus.cc:2229] T 2c6dfe6a92284c5f96094eb16c757222 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:13.935412 32089 raft_consensus.cc:2258] T 2c6dfe6a92284c5f96094eb16c757222 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:13.935463 32477 ts_tablet_manager.cc:1909] T 2c6dfe6a92284c5f96094eb16c757222 P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:13.935463 32088 ts_tablet_manager.cc:1909] T 2c6dfe6a92284c5f96094eb16c757222 P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:13.935766 32089 ts_tablet_manager.cc:1875] T 2c6dfe6a92284c5f96094eb16c757222 P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:13.936049 32477 tablet_replica.cc:330] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:13.936170 32477 raft_consensus.cc:2229] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:13.936172 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet 2c6dfe6a92284c5f96094eb16c757222 (table rebalance_test_table_10 [id=79c2e912ffdb4aae9cf78414231a842e]) successfully deleted
I20240502 20:46:13.936354 32477 raft_consensus.cc:2258] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:13.936802 32477 ts_tablet_manager.cc:1875] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:13.936923 31757 raft_consensus.cc:1261] T c18f9efa9c2043f6b2fad95ad9fd8296 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:13.937269 32089 ts_tablet_manager.cc:1888] T 2c6dfe6a92284c5f96094eb16c757222 P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:13.937326 32089 log.cc:1196] T 2c6dfe6a92284c5f96094eb16c757222 P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/2c6dfe6a92284c5f96094eb16c757222
I20240502 20:46:13.937383 355 consensus_queue.cc:1035] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:13.937486 32089 ts_tablet_manager.cc:1909] T 2c6dfe6a92284c5f96094eb16c757222 P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:13.937922 32089 tablet_replica.cc:330] T 9b609e0004524a67bb4eea7b3c5d58fb P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:13.938037 32089 raft_consensus.cc:2229] T 9b609e0004524a67bb4eea7b3c5d58fb P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:13.938046 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 2c6dfe6a92284c5f96094eb16c757222 (table rebalance_test_table_10 [id=79c2e912ffdb4aae9cf78414231a842e]) successfully deleted
I20240502 20:46:13.938311 32089 raft_consensus.cc:2258] T 9b609e0004524a67bb4eea7b3c5d58fb P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:13.938810 32089 ts_tablet_manager.cc:1875] T 9b609e0004524a67bb4eea7b3c5d58fb P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:13.938796 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet 2c6dfe6a92284c5f96094eb16c757222 (table rebalance_test_table_10 [id=79c2e912ffdb4aae9cf78414231a842e]) successfully deleted
I20240502 20:46:13.939199 31602 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c6dfe6a92284c5f96094eb16c757222 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:13 UTC) from {username='slave'} at 127.0.0.1:58346
I20240502 20:46:13.939379 508 tablet_replica.cc:330] T 2c6dfe6a92284c5f96094eb16c757222 P 315915efcf2b413ca7f9d874338e1b2d: stopping tablet replica
I20240502 20:46:13.939463 29557 tablet_service.cc:1514] Processing DeleteTablet for tablet 88a8cf3a902d4e5bbbaf47a5202d514a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:13 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:13.939492 508 raft_consensus.cc:2229] T 2c6dfe6a92284c5f96094eb16c757222 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:13.939589 508 raft_consensus.cc:2258] T 2c6dfe6a92284c5f96094eb16c757222 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:13.939622 29556 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b609e0004524a67bb4eea7b3c5d58fb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:13 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:13.939810 32088 tablet_replica.cc:330] T 88a8cf3a902d4e5bbbaf47a5202d514a P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:13.939927 32088 raft_consensus.cc:2229] T 88a8cf3a902d4e5bbbaf47a5202d514a P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:13.940007 29426 tablet_service.cc:1514] Processing DeleteTablet for tablet 88a8cf3a902d4e5bbbaf47a5202d514a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:13 UTC) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:13.940075 32088 raft_consensus.cc:2258] T 88a8cf3a902d4e5bbbaf47a5202d514a P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:13.940068 29314 raft_consensus.cc:1261] T c18f9efa9c2043f6b2fad95ad9fd8296 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:13.940147 508 ts_tablet_manager.cc:1875] T 2c6dfe6a92284c5f96094eb16c757222 P 315915efcf2b413ca7f9d874338e1b2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:13.940323 445 tablet_replica.cc:330] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:13.940450 445 raft_consensus.cc:2229] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:13.940510 32088 ts_tablet_manager.cc:1875] T 88a8cf3a902d4e5bbbaf47a5202d514a P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:13.940500 32571 consensus_queue.cc:1035] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:13.940621 445 raft_consensus.cc:2258] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:13.941053 445 ts_tablet_manager.cc:1875] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:13.941138 32477 ts_tablet_manager.cc:1888] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:13.941210 32477 log.cc:1196] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/9b609e0004524a67bb4eea7b3c5d58fb
I20240502 20:46:13.941454 32477 ts_tablet_manager.cc:1909] T 9b609e0004524a67bb4eea7b3c5d58fb P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:13.942070 508 ts_tablet_manager.cc:1888] T 2c6dfe6a92284c5f96094eb16c757222 P 315915efcf2b413ca7f9d874338e1b2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:13.942154 508 log.cc:1196] T 2c6dfe6a92284c5f96094eb16c757222 P 315915efcf2b413ca7f9d874338e1b2d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal/wals/2c6dfe6a92284c5f96094eb16c757222
I20240502 20:46:13.942409 508 ts_tablet_manager.cc:1909] T 2c6dfe6a92284c5f96094eb16c757222 P 315915efcf2b413ca7f9d874338e1b2d: Deleting consensus metadata
I20240502 20:46:13.942757 355 raft_consensus.cc:2932] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:13.942950 32088 ts_tablet_manager.cc:1888] T 88a8cf3a902d4e5bbbaf47a5202d514a P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:13.943019 32088 log.cc:1196] T 88a8cf3a902d4e5bbbaf47a5202d514a P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/88a8cf3a902d4e5bbbaf47a5202d514a
I20240502 20:46:13.943089 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet 9b609e0004524a67bb4eea7b3c5d58fb (table rebalance_test_table_10 [id=79c2e912ffdb4aae9cf78414231a842e]) successfully deleted
I20240502 20:46:13.943117 445 ts_tablet_manager.cc:1888] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:13.943174 445 log.cc:1196] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/88a8cf3a902d4e5bbbaf47a5202d514a
I20240502 20:46:13.943289 32088 ts_tablet_manager.cc:1909] T 88a8cf3a902d4e5bbbaf47a5202d514a P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:13.943400 445 ts_tablet_manager.cc:1909] T 88a8cf3a902d4e5bbbaf47a5202d514a P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting consensus metadata
I20240502 20:46:13.943903 32088 tablet_replica.cc:330] T 9b609e0004524a67bb4eea7b3c5d58fb P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:13.944614 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet 88a8cf3a902d4e5bbbaf47a5202d514a (table rebalance_test_table_10 [id=79c2e912ffdb4aae9cf78414231a842e]) successfully deleted
I20240502 20:46:13.944784 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet 88a8cf3a902d4e5bbbaf47a5202d514a (table rebalance_test_table_10 [id=79c2e912ffdb4aae9cf78414231a842e]) successfully deleted
I20240502 20:46:13.945575 32089 ts_tablet_manager.cc:1888] T 9b609e0004524a67bb4eea7b3c5d58fb P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:13.945616 31757 raft_consensus.cc:3037] T 28d6a4d9c92141f794d6ee04335cf9b8 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:13.945650 32089 log.cc:1196] T 9b609e0004524a67bb4eea7b3c5d58fb P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/9b609e0004524a67bb4eea7b3c5d58fb
I20240502 20:46:13.945519 31756 raft_consensus.cc:2932] T c18f9efa9c2043f6b2fad95ad9fd8296 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:13.945873 32089 ts_tablet_manager.cc:1909] T 9b609e0004524a67bb4eea7b3c5d58fb P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:13.945780 29035 catalog_manager.cc:5484] T c18f9efa9c2043f6b2fad95ad9fd8296 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index 4 to 5, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) evicted. New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:13.944020 32088 raft_consensus.cc:2229] T 9b609e0004524a67bb4eea7b3c5d58fb P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:13.947645 32088 raft_consensus.cc:2258] T 9b609e0004524a67bb4eea7b3c5d58fb P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:13.947691 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 9b609e0004524a67bb4eea7b3c5d58fb (table rebalance_test_table_10 [id=79c2e912ffdb4aae9cf78414231a842e]) successfully deleted
I20240502 20:46:13.948132 32088 ts_tablet_manager.cc:1875] T 9b609e0004524a67bb4eea7b3c5d58fb P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:13.948959 29556 tablet_service.cc:1514] Processing DeleteTablet for tablet c18f9efa9c2043f6b2fad95ad9fd8296 with delete_type TABLET_DATA_TOMBSTONED (TS db17f26a73f3447b92aa9a947e3a7057 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:13.948985 32708 consensus_queue.cc:1035] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:13.949524 29314 raft_consensus.cc:2932] T c18f9efa9c2043f6b2fad95ad9fd8296 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:13.950186 32088 ts_tablet_manager.cc:1888] T 9b609e0004524a67bb4eea7b3c5d58fb P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:13.950237 32088 log.cc:1196] T 9b609e0004524a67bb4eea7b3c5d58fb P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/9b609e0004524a67bb4eea7b3c5d58fb
I20240502 20:46:13.950398 32088 ts_tablet_manager.cc:1909] T 9b609e0004524a67bb4eea7b3c5d58fb P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:13.951010 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet 9b609e0004524a67bb4eea7b3c5d58fb (table rebalance_test_table_10 [id=79c2e912ffdb4aae9cf78414231a842e]) successfully deleted
I20240502 20:46:13.951350 32088 tablet_replica.cc:330] T c18f9efa9c2043f6b2fad95ad9fd8296 P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:13.951565 32088 raft_consensus.cc:2229] T c18f9efa9c2043f6b2fad95ad9fd8296 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:13.951756 32088 raft_consensus.cc:2258] T c18f9efa9c2043f6b2fad95ad9fd8296 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:13.952376 29027 catalog_manager.cc:4830] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): tablet 2c6dfe6a92284c5f96094eb16c757222 (table rebalance_test_table_10 [id=79c2e912ffdb4aae9cf78414231a842e]) successfully deleted
I20240502 20:46:13.952525 32088 ts_tablet_manager.cc:1875] T c18f9efa9c2043f6b2fad95ad9fd8296 P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:13.954860 32708 consensus_queue.cc:1035] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240502 20:46:13.957515 32088 ts_tablet_manager.cc:1888] T c18f9efa9c2043f6b2fad95ad9fd8296 P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 20:46:13.957623 32088 log.cc:1196] T c18f9efa9c2043f6b2fad95ad9fd8296 P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/c18f9efa9c2043f6b2fad95ad9fd8296
I20240502 20:46:13.958148 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet c18f9efa9c2043f6b2fad95ad9fd8296 (table rebalance_test_table_19 [id=5fd0afffa41641b9b8de6545456ff39a]) successfully deleted
I20240502 20:46:13.981194 32108 consensus_queue.cc:1035] T b718cecdc692408bbeea91bc9fe21fcb P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:13.984864 31626 raft_consensus.cc:3037] T b718cecdc692408bbeea91bc9fe21fcb P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:13.984936 32108 raft_consensus.cc:482] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:13.985033 32108 raft_consensus.cc:504] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.985414 32108 leader_election.cc:288] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:13.985771 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a03d782933b4c8bb95a97c142b22388" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:13.985913 31757 raft_consensus.cc:2454] T 2a03d782933b4c8bb95a97c142b22388 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:13.986243 31428 leader_election.cc:302] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:13.986634 487 raft_consensus.cc:2781] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:13.986697 487 raft_consensus.cc:482] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:13.986735 487 raft_consensus.cc:3037] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:13.987658 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a03d782933b4c8bb95a97c142b22388" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:13.987694 32108 consensus_queue.cc:1035] T b718cecdc692408bbeea91bc9fe21fcb P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:13.987722 487 raft_consensus.cc:504] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.987785 31626 raft_consensus.cc:2454] T 2a03d782933b4c8bb95a97c142b22388 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:13.988049 487 leader_election.cc:288] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:13.988370 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a03d782933b4c8bb95a97c142b22388" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:13.988469 31757 raft_consensus.cc:3037] T 2a03d782933b4c8bb95a97c142b22388 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:13.988855 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a03d782933b4c8bb95a97c142b22388" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:13.988955 31625 raft_consensus.cc:3037] T 2a03d782933b4c8bb95a97c142b22388 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:13.989408 31757 raft_consensus.cc:2454] T 2a03d782933b4c8bb95a97c142b22388 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:13.989737 31428 leader_election.cc:302] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:13.989811 31625 raft_consensus.cc:2454] T 2a03d782933b4c8bb95a97c142b22388 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:13.989892 487 raft_consensus.cc:2781] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:13.989956 487 raft_consensus.cc:686] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Becoming Leader. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Running, Role: LEADER
I20240502 20:46:13.990082 487 consensus_queue.cc:237] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:13.995551 29033 catalog_manager.cc:5484] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69). New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:14.047084 29295 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:14.049664 29162 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:14.049765 31474 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:14.051893 29556 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:14.062451 31602 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:46:14.065225 29426 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:14.081487 31868 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:46:14.093549 31737 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:14.122673 29036 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:14.127010 31757 raft_consensus.cc:1261] T 2a03d782933b4c8bb95a97c142b22388 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:14.127010 31626 raft_consensus.cc:1261] T 2a03d782933b4c8bb95a97c142b22388 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:14.127497 487 consensus_queue.cc:1035] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:14.127645 32108 consensus_queue.cc:1035] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:14.130800 29034 catalog_manager.cc:5576] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787) does not have the latest schema for tablet 2a03d782933b4c8bb95a97c142b22388 (table rebalancer_dynamic_table [id=d409c40457214940a8c89c7071ba7749]). Expected version 1 got 0
I20240502 20:46:14.131116 29036 catalog_manager.cc:5576] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751) does not have the latest schema for tablet 2a03d782933b4c8bb95a97c142b22388 (table rebalancer_dynamic_table [id=d409c40457214940a8c89c7071ba7749]). Expected version 1 got 0
I20240502 20:46:14.132143 661 tablet.cc:1722] T 2a03d782933b4c8bb95a97c142b22388 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:14.132241 660 tablet.cc:1722] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:14.132570 660 tablet.cc:1620] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:14.132678 661 tablet.cc:1620] T 2a03d782933b4c8bb95a97c142b22388 P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:14.133354 29036 catalog_manager.cc:5576] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751) does not have the latest schema for tablet 2a03d782933b4c8bb95a97c142b22388 (table rebalancer_dynamic_table [id=d409c40457214940a8c89c7071ba7749]). Expected version 1 got 0
I20240502 20:46:14.135708 29036 catalog_manager.cc:5576] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787) does not have the latest schema for tablet 2a03d782933b4c8bb95a97c142b22388 (table rebalancer_dynamic_table [id=d409c40457214940a8c89c7071ba7749]). Expected version 1 got 0
I20240502 20:46:14.135946 29175 consensus_queue.cc:237] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:14.137308 31494 raft_consensus.cc:1261] T 4fd651e80e004dc58d7769f227af58a8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:14.137704 642 consensus_queue.cc:1035] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:14.137758 29315 raft_consensus.cc:1261] T 4fd651e80e004dc58d7769f227af58a8 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:14.138111 642 consensus_queue.cc:1035] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:14.138877 29034 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=d409c40457214940a8c89c7071ba7749] alter complete (version 1)
I20240502 20:46:14.140775 662 tablet.cc:1722] T 2a03d782933b4c8bb95a97c142b22388 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:14.141101 662 tablet.cc:1620] T 2a03d782933b4c8bb95a97c142b22388 P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:14.141129 642 raft_consensus.cc:2932] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:14.141567 31494 raft_consensus.cc:2932] T 4fd651e80e004dc58d7769f227af58a8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:14.143852 29315 raft_consensus.cc:2932] T 4fd651e80e004dc58d7769f227af58a8 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:14.144532 29034 catalog_manager.cc:5484] T 4fd651e80e004dc58d7769f227af58a8 P 7796ea3fba294d17a34386873969431d reported cstate change: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:14.147403 29426 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fd651e80e004dc58d7769f227af58a8 with delete_type TABLET_DATA_TOMBSTONED (TS dbcdc6ac635b44b185c3eba63f66e9d6 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:14.147886 445 tablet_replica.cc:330] T 4fd651e80e004dc58d7769f227af58a8 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:14.148046 445 raft_consensus.cc:2229] T 4fd651e80e004dc58d7769f227af58a8 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:14.148177 445 raft_consensus.cc:2258] T 4fd651e80e004dc58d7769f227af58a8 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:14.148707 445 ts_tablet_manager.cc:1875] T 4fd651e80e004dc58d7769f227af58a8 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:14.150738 445 ts_tablet_manager.cc:1888] T 4fd651e80e004dc58d7769f227af58a8 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 20:46:14.150817 445 log.cc:1196] T 4fd651e80e004dc58d7769f227af58a8 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/4fd651e80e004dc58d7769f227af58a8
I20240502 20:46:14.151274 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet 4fd651e80e004dc58d7769f227af58a8 (table rebalance_test_table_3 [id=aec26ee01a7a4b05887071d6b1783fb0]) successfully deleted
I20240502 20:46:14.152626 29036 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:14.159811 31602 tablet_service.cc:1467] Processing CreateTablet for tablet 03db15f1ebfb4e4ea271d6bb5c76110d (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=9659763c7b3a4e2f8ffa64e3e9c5e095]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:14.160123 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03db15f1ebfb4e4ea271d6bb5c76110d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:14.160735 29294 tablet_service.cc:1467] Processing CreateTablet for tablet f3c220fbd7df4cefb467ad88e99941b8 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=9659763c7b3a4e2f8ffa64e3e9c5e095]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:14.160893 31737 tablet_service.cc:1467] Processing CreateTablet for tablet 03db15f1ebfb4e4ea271d6bb5c76110d (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=9659763c7b3a4e2f8ffa64e3e9c5e095]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:14.160992 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3c220fbd7df4cefb467ad88e99941b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:14.161132 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03db15f1ebfb4e4ea271d6bb5c76110d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:14.161881 561 tablet_bootstrap.cc:492] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:14.162339 31606 tablet_service.cc:1467] Processing CreateTablet for tablet f3c220fbd7df4cefb467ad88e99941b8 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=9659763c7b3a4e2f8ffa64e3e9c5e095]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:14.162608 31606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3c220fbd7df4cefb467ad88e99941b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:14.162801 561 tablet_bootstrap.cc:654] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:14.163204 356 tablet_bootstrap.cc:492] T 03db15f1ebfb4e4ea271d6bb5c76110d P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:14.163973 356 tablet_bootstrap.cc:654] T 03db15f1ebfb4e4ea271d6bb5c76110d P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:14.164660 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 7e0f811a7bd24ab395364e9461144558 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=9659763c7b3a4e2f8ffa64e3e9c5e095]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:14.164912 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e0f811a7bd24ab395364e9461144558. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:14.165095 678 tablet_bootstrap.cc:492] T f3c220fbd7df4cefb467ad88e99941b8 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:14.165681 356 tablet_bootstrap.cc:492] T 03db15f1ebfb4e4ea271d6bb5c76110d P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:14.165760 356 ts_tablet_manager.cc:1367] T 03db15f1ebfb4e4ea271d6bb5c76110d P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 20:46:14.165896 678 tablet_bootstrap.cc:654] T f3c220fbd7df4cefb467ad88e99941b8 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:14.166208 356 raft_consensus.cc:348] T 03db15f1ebfb4e4ea271d6bb5c76110d P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:14.166316 356 raft_consensus.cc:374] T 03db15f1ebfb4e4ea271d6bb5c76110d P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:14.166375 356 raft_consensus.cc:729] T 03db15f1ebfb4e4ea271d6bb5c76110d P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:14.166520 356 consensus_queue.cc:260] T 03db15f1ebfb4e4ea271d6bb5c76110d P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:14.166790 356 ts_tablet_manager.cc:1398] T 03db15f1ebfb4e4ea271d6bb5c76110d P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:14.167138 31868 tablet_service.cc:1467] Processing CreateTablet for tablet 03db15f1ebfb4e4ea271d6bb5c76110d (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=9659763c7b3a4e2f8ffa64e3e9c5e095]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:14.167331 678 tablet_bootstrap.cc:492] T f3c220fbd7df4cefb467ad88e99941b8 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:14.167407 678 ts_tablet_manager.cc:1367] T f3c220fbd7df4cefb467ad88e99941b8 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:14.167438 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03db15f1ebfb4e4ea271d6bb5c76110d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:14.167876 678 raft_consensus.cc:348] T f3c220fbd7df4cefb467ad88e99941b8 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:14.168004 678 raft_consensus.cc:374] T f3c220fbd7df4cefb467ad88e99941b8 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:14.168035 678 raft_consensus.cc:729] T f3c220fbd7df4cefb467ad88e99941b8 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:14.168154 678 consensus_queue.cc:260] T f3c220fbd7df4cefb467ad88e99941b8 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:14.168429 678 ts_tablet_manager.cc:1398] T f3c220fbd7df4cefb467ad88e99941b8 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:14.168586 678 tablet_bootstrap.cc:492] T 7e0f811a7bd24ab395364e9461144558 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:14.169659 678 tablet_bootstrap.cc:654] T 7e0f811a7bd24ab395364e9461144558 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:14.170516 31737 tablet_service.cc:1467] Processing CreateTablet for tablet 7e0f811a7bd24ab395364e9461144558 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=9659763c7b3a4e2f8ffa64e3e9c5e095]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:14.170859 678 tablet_bootstrap.cc:492] T 7e0f811a7bd24ab395364e9461144558 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:14.170935 678 ts_tablet_manager.cc:1367] T 7e0f811a7bd24ab395364e9461144558 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:14.171195 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e0f811a7bd24ab395364e9461144558. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:14.171171 29162 tablet_service.cc:1467] Processing CreateTablet for tablet 7e0f811a7bd24ab395364e9461144558 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=9659763c7b3a4e2f8ffa64e3e9c5e095]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:14.171432 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e0f811a7bd24ab395364e9461144558. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:14.172175 678 raft_consensus.cc:348] T 7e0f811a7bd24ab395364e9461144558 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:14.172293 678 raft_consensus.cc:374] T 7e0f811a7bd24ab395364e9461144558 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:14.172343 678 raft_consensus.cc:729] T 7e0f811a7bd24ab395364e9461144558 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:14.173269 356 tablet_bootstrap.cc:492] T 7e0f811a7bd24ab395364e9461144558 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:14.174168 356 tablet_bootstrap.cc:654] T 7e0f811a7bd24ab395364e9461144558 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:14.172483 678 consensus_queue.cc:260] T 7e0f811a7bd24ab395364e9461144558 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:14.175177 678 ts_tablet_manager.cc:1398] T 7e0f811a7bd24ab395364e9461144558 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:14.175602 31474 tablet_service.cc:1467] Processing CreateTablet for tablet f3c220fbd7df4cefb467ad88e99941b8 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=9659763c7b3a4e2f8ffa64e3e9c5e095]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:14.175896 31474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3c220fbd7df4cefb467ad88e99941b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:14.177985 32108 consensus_queue.cc:1035] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:14.178403 436 tablet_bootstrap.cc:492] T f3c220fbd7df4cefb467ad88e99941b8 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:14.179244 436 tablet_bootstrap.cc:654] T f3c220fbd7df4cefb467ad88e99941b8 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:14.182418 356 tablet_bootstrap.cc:492] T 7e0f811a7bd24ab395364e9461144558 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:14.182556 356 ts_tablet_manager.cc:1367] T 7e0f811a7bd24ab395364e9461144558 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20240502 20:46:14.183233 356 raft_consensus.cc:348] T 7e0f811a7bd24ab395364e9461144558 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:14.183352 356 raft_consensus.cc:374] T 7e0f811a7bd24ab395364e9461144558 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:14.183384 356 raft_consensus.cc:729] T 7e0f811a7bd24ab395364e9461144558 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:14.183514 356 consensus_queue.cc:260] T 7e0f811a7bd24ab395364e9461144558 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:14.183795 356 ts_tablet_manager.cc:1398] T 7e0f811a7bd24ab395364e9461144558 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:14.185976 436 tablet_bootstrap.cc:492] T f3c220fbd7df4cefb467ad88e99941b8 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:14.186062 436 ts_tablet_manager.cc:1367] T f3c220fbd7df4cefb467ad88e99941b8 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 20:46:14.186089 686 tablet_bootstrap.cc:492] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:14.187469 686 tablet_bootstrap.cc:654] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:14.187589 436 raft_consensus.cc:348] T f3c220fbd7df4cefb467ad88e99941b8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:14.187706 436 raft_consensus.cc:374] T f3c220fbd7df4cefb467ad88e99941b8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:14.187732 436 raft_consensus.cc:729] T f3c220fbd7df4cefb467ad88e99941b8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:14.187839 436 consensus_queue.cc:260] T f3c220fbd7df4cefb467ad88e99941b8 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:14.188087 436 ts_tablet_manager.cc:1398] T f3c220fbd7df4cefb467ad88e99941b8 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240502 20:46:14.189595 686 tablet_bootstrap.cc:492] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:14.189697 686 ts_tablet_manager.cc:1367] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:14.190099 686 raft_consensus.cc:348] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:14.190199 686 raft_consensus.cc:374] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:14.190229 686 raft_consensus.cc:729] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:14.190348 686 consensus_queue.cc:260] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:14.190625 686 ts_tablet_manager.cc:1398] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:14.193361 681 tablet_bootstrap.cc:492] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:14.194588 681 tablet_bootstrap.cc:654] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:14.195777 681 tablet_bootstrap.cc:492] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:14.195865 681 ts_tablet_manager.cc:1367] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 20:46:14.196242 681 raft_consensus.cc:348] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:14.196337 681 raft_consensus.cc:374] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:14.196367 681 raft_consensus.cc:729] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:14.196595 681 consensus_queue.cc:260] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:14.196977 681 ts_tablet_manager.cc:1398] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:14.198643 561 tablet_bootstrap.cc:492] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:14.198755 561 ts_tablet_manager.cc:1367] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.037s user 0.002s sys 0.000s
I20240502 20:46:14.199190 561 raft_consensus.cc:348] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:14.199290 561 raft_consensus.cc:374] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:14.199321 561 raft_consensus.cc:729] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:14.199438 561 consensus_queue.cc:260] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:14.199720 561 ts_tablet_manager.cc:1398] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:14.199890 561 tablet_bootstrap.cc:492] T f3c220fbd7df4cefb467ad88e99941b8 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:14.200762 561 tablet_bootstrap.cc:654] T f3c220fbd7df4cefb467ad88e99941b8 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:14.200722 487 consensus_queue.cc:1035] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:14.201764 561 tablet_bootstrap.cc:492] T f3c220fbd7df4cefb467ad88e99941b8 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:14.201835 561 ts_tablet_manager.cc:1367] T f3c220fbd7df4cefb467ad88e99941b8 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:14.202270 561 raft_consensus.cc:348] T f3c220fbd7df4cefb467ad88e99941b8 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:14.202354 561 raft_consensus.cc:374] T f3c220fbd7df4cefb467ad88e99941b8 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:14.202383 561 raft_consensus.cc:729] T f3c220fbd7df4cefb467ad88e99941b8 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:14.202538 561 consensus_queue.cc:260] T f3c220fbd7df4cefb467ad88e99941b8 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:14.202826 561 ts_tablet_manager.cc:1398] T f3c220fbd7df4cefb467ad88e99941b8 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:14.242372 29036 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:14.248723 660 tablet.cc:1722] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:14.249061 660 tablet.cc:1620] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:14.249894 662 tablet.cc:1722] T 2a03d782933b4c8bb95a97c142b22388 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:14.250193 662 tablet.cc:1620] T 2a03d782933b4c8bb95a97c142b22388 P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:14.251098 661 tablet.cc:1722] T 2a03d782933b4c8bb95a97c142b22388 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:14.251399 661 tablet.cc:1620] T 2a03d782933b4c8bb95a97c142b22388 P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:14.253764 605 raft_consensus.cc:482] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:14.253862 605 raft_consensus.cc:504] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:14.254245 605 leader_election.cc:288] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:14.254743 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03db15f1ebfb4e4ea271d6bb5c76110d" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:14.254871 31757 raft_consensus.cc:2454] T 03db15f1ebfb4e4ea271d6bb5c76110d P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:14.255008 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03db15f1ebfb4e4ea271d6bb5c76110d" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:14.255158 31888 raft_consensus.cc:2454] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:14.255234 31559 leader_election.cc:302] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:14.255373 605 raft_consensus.cc:2781] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:14.255417 605 raft_consensus.cc:482] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:14.255451 605 raft_consensus.cc:3037] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:14.256384 605 raft_consensus.cc:504] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:14.256703 605 leader_election.cc:288] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Requested vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:14.257161 32191 raft_consensus.cc:482] T f3c220fbd7df4cefb467ad88e99941b8 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:14.257273 32191 raft_consensus.cc:504] T f3c220fbd7df4cefb467ad88e99941b8 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:14.257665 32191 leader_election.cc:288] T f3c220fbd7df4cefb467ad88e99941b8 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:14.257709 29036 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=d409c40457214940a8c89c7071ba7749] alter complete (version 2)
I20240502 20:46:14.257750 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03db15f1ebfb4e4ea271d6bb5c76110d" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:14.257863 31888 raft_consensus.cc:3037] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:14.258121 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3c220fbd7df4cefb467ad88e99941b8" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:14.258245 31494 raft_consensus.cc:2454] T f3c220fbd7df4cefb467ad88e99941b8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:14.258250 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3c220fbd7df4cefb467ad88e99941b8" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:14.258358 31626 raft_consensus.cc:2454] T f3c220fbd7df4cefb467ad88e99941b8 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:14.258574 29250 leader_election.cc:302] T f3c220fbd7df4cefb467ad88e99941b8 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 2a9c1af8cf814030bde56b67ca5c2fbe; no voters:
I20240502 20:46:14.258796 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03db15f1ebfb4e4ea271d6bb5c76110d" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:14.258875 31888 raft_consensus.cc:2454] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:14.258915 31757 raft_consensus.cc:3037] T 03db15f1ebfb4e4ea271d6bb5c76110d P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:14.259202 32191 raft_consensus.cc:2781] T f3c220fbd7df4cefb467ad88e99941b8 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:14.259271 32191 raft_consensus.cc:482] T f3c220fbd7df4cefb467ad88e99941b8 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:14.259320 32191 raft_consensus.cc:3037] T f3c220fbd7df4cefb467ad88e99941b8 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:14.259366 31560 leader_election.cc:302] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:14.259909 605 raft_consensus.cc:2781] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:14.259990 605 raft_consensus.cc:686] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Becoming Leader. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Running, Role: LEADER
I20240502 20:46:14.260114 605 consensus_queue.cc:237] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:14.260152 32191 raft_consensus.cc:504] T f3c220fbd7df4cefb467ad88e99941b8 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:14.260448 32191 leader_election.cc:288] T f3c220fbd7df4cefb467ad88e99941b8 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:14.260731 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3c220fbd7df4cefb467ad88e99941b8" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:14.260829 31626 raft_consensus.cc:3037] T f3c220fbd7df4cefb467ad88e99941b8 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:14.261544 31626 raft_consensus.cc:2454] T f3c220fbd7df4cefb467ad88e99941b8 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:14.261754 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3c220fbd7df4cefb467ad88e99941b8" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:14.261797 29249 leader_election.cc:302] T f3c220fbd7df4cefb467ad88e99941b8 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 315915efcf2b413ca7f9d874338e1b2d; no voters:
I20240502 20:46:14.261862 31494 raft_consensus.cc:3037] T f3c220fbd7df4cefb467ad88e99941b8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:14.261977 32191 raft_consensus.cc:2781] T f3c220fbd7df4cefb467ad88e99941b8 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:14.262149 32191 raft_consensus.cc:686] T f3c220fbd7df4cefb467ad88e99941b8 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:14.262287 32191 consensus_queue.cc:237] T f3c220fbd7df4cefb467ad88e99941b8 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:14.262657 31494 raft_consensus.cc:2454] T f3c220fbd7df4cefb467ad88e99941b8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:14.263617 29040 catalog_manager.cc:5484] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70). New cstate: current_term: 1 leader_uuid: "315915efcf2b413ca7f9d874338e1b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:14.264454 29034 catalog_manager.cc:5484] T f3c220fbd7df4cefb467ad88e99941b8 P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:14.283434 31757 raft_consensus.cc:2454] T 03db15f1ebfb4e4ea271d6bb5c76110d P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:14.349233 32213 raft_consensus.cc:482] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:14.349354 32213 raft_consensus.cc:504] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:14.349761 32213 leader_election.cc:288] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:14.350129 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e0f811a7bd24ab395364e9461144558" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:14.350149 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e0f811a7bd24ab395364e9461144558" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:14.350286 29315 raft_consensus.cc:2454] T 7e0f811a7bd24ab395364e9461144558 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:14.350298 31757 raft_consensus.cc:2454] T 7e0f811a7bd24ab395364e9461144558 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:14.350585 29117 leader_election.cc:302] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:14.350886 32213 raft_consensus.cc:2781] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:14.350948 32213 raft_consensus.cc:482] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:14.350994 32213 raft_consensus.cc:3037] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:14.351851 32213 raft_consensus.cc:504] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:14.352241 32213 leader_election.cc:288] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:14.352648 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e0f811a7bd24ab395364e9461144558" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:14.352751 31757 raft_consensus.cc:3037] T 7e0f811a7bd24ab395364e9461144558 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:14.353685 31757 raft_consensus.cc:2454] T 7e0f811a7bd24ab395364e9461144558 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:14.353915 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e0f811a7bd24ab395364e9461144558" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:14.353956 29117 leader_election.cc:302] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:14.354032 29315 raft_consensus.cc:3037] T 7e0f811a7bd24ab395364e9461144558 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:14.354118 32213 raft_consensus.cc:2781] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:14.354192 32213 raft_consensus.cc:686] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:14.354315 32213 consensus_queue.cc:237] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:14.354976 29315 raft_consensus.cc:2454] T 7e0f811a7bd24ab395364e9461144558 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:14.356527 29040 catalog_manager.cc:5484] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:14.359925 29040 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:14.368132 660 tablet.cc:1722] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:14.368883 661 tablet.cc:1722] T 2a03d782933b4c8bb95a97c142b22388 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:14.369166 662 tablet.cc:1722] T 2a03d782933b4c8bb95a97c142b22388 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:14.376742 29034 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=d409c40457214940a8c89c7071ba7749] alter complete (version 3)
I20240502 20:46:14.382228 29162 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:14.383325 31606 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:46:14.397380 29295 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:14.400066 29556 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:14.401883 31474 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:14.402673 29426 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:14.416082 31868 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:46:14.424157 31737 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:14.442422 29036 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalance_test_table_12" } modify_external_catalogs: true
I20240502 20:46:14.442615 29036 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalance_test_table_12" } modify_external_catalogs: true
I20240502 20:46:14.445029 29036 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 4 replicas of tablet b0e8b1909df34fce92f94d6af5f5c3b6
I20240502 20:46:14.445411 29036 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet f65eb8cde25e483b9c790740ccfd357c
I20240502 20:46:14.445673 29036 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 4 replicas of tablet 94bdf135dd3a47d39e7b76dbfd524386
I20240502 20:46:14.445741 29295 tablet_service.cc:1514] Processing DeleteTablet for tablet b0e8b1909df34fce92f94d6af5f5c3b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:14 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:14.445940 31606 tablet_service.cc:1514] Processing DeleteTablet for tablet b0e8b1909df34fce92f94d6af5f5c3b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:14 UTC) from {username='slave'} at 127.0.0.1:58346
I20240502 20:46:14.446172 29162 tablet_service.cc:1514] Processing DeleteTablet for tablet b0e8b1909df34fce92f94d6af5f5c3b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:14 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:14.446215 508 tablet_replica.cc:330] T b0e8b1909df34fce92f94d6af5f5c3b6 P 315915efcf2b413ca7f9d874338e1b2d: stopping tablet replica
I20240502 20:46:14.446264 29556 tablet_service.cc:1514] Processing DeleteTablet for tablet f65eb8cde25e483b9c790740ccfd357c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:14 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:14.446269 29426 tablet_service.cc:1514] Processing DeleteTablet for tablet b0e8b1909df34fce92f94d6af5f5c3b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:14 UTC) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:14.446342 508 raft_consensus.cc:2229] T b0e8b1909df34fce92f94d6af5f5c3b6 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:14.446372 32089 tablet_replica.cc:330] T b0e8b1909df34fce92f94d6af5f5c3b6 P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:14.446427 32088 tablet_replica.cc:330] T f65eb8cde25e483b9c790740ccfd357c P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:14.446449 508 raft_consensus.cc:2258] T b0e8b1909df34fce92f94d6af5f5c3b6 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:14.446519 32089 raft_consensus.cc:2229] T b0e8b1909df34fce92f94d6af5f5c3b6 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:14.446563 445 tablet_replica.cc:330] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:14.446565 32088 raft_consensus.cc:2229] T f65eb8cde25e483b9c790740ccfd357c P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:14.446655 32089 raft_consensus.cc:2258] T b0e8b1909df34fce92f94d6af5f5c3b6 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:14.446702 32088 raft_consensus.cc:2258] T f65eb8cde25e483b9c790740ccfd357c P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:14.446712 445 raft_consensus.cc:2229] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:14.446941 445 raft_consensus.cc:2258] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:14.447006 508 ts_tablet_manager.cc:1875] T b0e8b1909df34fce92f94d6af5f5c3b6 P 315915efcf2b413ca7f9d874338e1b2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:14.447118 32089 ts_tablet_manager.cc:1875] T b0e8b1909df34fce92f94d6af5f5c3b6 P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:14.447196 32088 ts_tablet_manager.cc:1875] T f65eb8cde25e483b9c790740ccfd357c P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:14.447397 445 ts_tablet_manager.cc:1875] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:14.449214 508 ts_tablet_manager.cc:1888] T b0e8b1909df34fce92f94d6af5f5c3b6 P 315915efcf2b413ca7f9d874338e1b2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:14.449223 32089 ts_tablet_manager.cc:1888] T b0e8b1909df34fce92f94d6af5f5c3b6 P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:14.449283 508 log.cc:1196] T b0e8b1909df34fce92f94d6af5f5c3b6 P 315915efcf2b413ca7f9d874338e1b2d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal/wals/b0e8b1909df34fce92f94d6af5f5c3b6
I20240502 20:46:14.449283 32089 log.cc:1196] T b0e8b1909df34fce92f94d6af5f5c3b6 P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/b0e8b1909df34fce92f94d6af5f5c3b6
I20240502 20:46:14.449331 32088 ts_tablet_manager.cc:1888] T f65eb8cde25e483b9c790740ccfd357c P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:14.449331 445 ts_tablet_manager.cc:1888] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:14.449398 32088 log.cc:1196] T f65eb8cde25e483b9c790740ccfd357c P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/f65eb8cde25e483b9c790740ccfd357c
I20240502 20:46:14.449415 445 log.cc:1196] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/b0e8b1909df34fce92f94d6af5f5c3b6
I20240502 20:46:14.449537 508 ts_tablet_manager.cc:1909] T b0e8b1909df34fce92f94d6af5f5c3b6 P 315915efcf2b413ca7f9d874338e1b2d: Deleting consensus metadata
I20240502 20:46:14.449532 32089 ts_tablet_manager.cc:1909] T b0e8b1909df34fce92f94d6af5f5c3b6 P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:14.449642 445 ts_tablet_manager.cc:1909] T b0e8b1909df34fce92f94d6af5f5c3b6 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting consensus metadata
I20240502 20:46:14.449647 32088 ts_tablet_manager.cc:1909] T f65eb8cde25e483b9c790740ccfd357c P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:14.450357 738 tablet_replica.cc:330] T b0e8b1909df34fce92f94d6af5f5c3b6 P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:14.450659 29162 tablet_service.cc:1514] Processing DeleteTablet for tablet f65eb8cde25e483b9c790740ccfd357c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:14 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:14.450814 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet b0e8b1909df34fce92f94d6af5f5c3b6 (table rebalance_test_table_12 [id=e2320262ebe84e399b45c01bf9a7d73d]) successfully deleted
I20240502 20:46:14.450927 32089 tablet_replica.cc:330] T f65eb8cde25e483b9c790740ccfd357c P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:14.451022 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet b0e8b1909df34fce92f94d6af5f5c3b6 (table rebalance_test_table_12 [id=e2320262ebe84e399b45c01bf9a7d73d]) successfully deleted
I20240502 20:46:14.451052 32089 raft_consensus.cc:2229] T f65eb8cde25e483b9c790740ccfd357c P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:14.451225 29295 tablet_service.cc:1514] Processing DeleteTablet for tablet f65eb8cde25e483b9c790740ccfd357c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:14 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:14.451256 32089 raft_consensus.cc:2258] T f65eb8cde25e483b9c790740ccfd357c P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:14.451298 29294 tablet_service.cc:1514] Processing DeleteTablet for tablet 94bdf135dd3a47d39e7b76dbfd524386 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:14 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:14.451507 738 raft_consensus.cc:2229] T b0e8b1909df34fce92f94d6af5f5c3b6 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:14.451627 29426 tablet_service.cc:1514] Processing DeleteTablet for tablet 94bdf135dd3a47d39e7b76dbfd524386 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:14 UTC) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:14.451776 738 raft_consensus.cc:2258] T b0e8b1909df34fce92f94d6af5f5c3b6 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:14.451838 29027 catalog_manager.cc:4830] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): tablet b0e8b1909df34fce92f94d6af5f5c3b6 (table rebalance_test_table_12 [id=e2320262ebe84e399b45c01bf9a7d73d]) successfully deleted
I20240502 20:46:14.451965 32089 ts_tablet_manager.cc:1875] T f65eb8cde25e483b9c790740ccfd357c P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:14.452003 445 tablet_replica.cc:330] T 94bdf135dd3a47d39e7b76dbfd524386 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:14.451999 31868 tablet_service.cc:1514] Processing DeleteTablet for tablet 94bdf135dd3a47d39e7b76dbfd524386 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:14 UTC) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:14.452164 445 raft_consensus.cc:2229] T 94bdf135dd3a47d39e7b76dbfd524386 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:14.452224 29556 tablet_service.cc:1514] Processing DeleteTablet for tablet 94bdf135dd3a47d39e7b76dbfd524386 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:14 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:14.452348 445 raft_consensus.cc:2258] T 94bdf135dd3a47d39e7b76dbfd524386 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:14.452692 738 ts_tablet_manager.cc:1875] T b0e8b1909df34fce92f94d6af5f5c3b6 P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:14.452795 445 ts_tablet_manager.cc:1875] T 94bdf135dd3a47d39e7b76dbfd524386 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:14.452817 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet f65eb8cde25e483b9c790740ccfd357c (table rebalance_test_table_12 [id=e2320262ebe84e399b45c01bf9a7d73d]) successfully deleted
I20240502 20:46:14.453050 32088 tablet_replica.cc:330] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:14.453214 32088 raft_consensus.cc:2229] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:14.453472 32088 raft_consensus.cc:2258] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:14.453941 32089 ts_tablet_manager.cc:1888] T f65eb8cde25e483b9c790740ccfd357c P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:14.454355 32089 log.cc:1196] T f65eb8cde25e483b9c790740ccfd357c P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/f65eb8cde25e483b9c790740ccfd357c
I20240502 20:46:14.454658 445 ts_tablet_manager.cc:1888] T 94bdf135dd3a47d39e7b76dbfd524386 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:14.454694 32089 ts_tablet_manager.cc:1909] T f65eb8cde25e483b9c790740ccfd357c P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:14.454777 445 log.cc:1196] T 94bdf135dd3a47d39e7b76dbfd524386 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/94bdf135dd3a47d39e7b76dbfd524386
I20240502 20:46:14.455137 32088 ts_tablet_manager.cc:1875] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:14.455222 445 ts_tablet_manager.cc:1909] T 94bdf135dd3a47d39e7b76dbfd524386 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting consensus metadata
I20240502 20:46:14.455343 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet f65eb8cde25e483b9c790740ccfd357c (table rebalance_test_table_12 [id=e2320262ebe84e399b45c01bf9a7d73d]) successfully deleted
I20240502 20:46:14.455847 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet 94bdf135dd3a47d39e7b76dbfd524386 (table rebalance_test_table_12 [id=e2320262ebe84e399b45c01bf9a7d73d]) successfully deleted
I20240502 20:46:14.456666 738 ts_tablet_manager.cc:1888] T b0e8b1909df34fce92f94d6af5f5c3b6 P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:14.456737 738 log.cc:1196] T b0e8b1909df34fce92f94d6af5f5c3b6 P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/b0e8b1909df34fce92f94d6af5f5c3b6
I20240502 20:46:14.457049 738 ts_tablet_manager.cc:1909] T b0e8b1909df34fce92f94d6af5f5c3b6 P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:14.457154 32088 ts_tablet_manager.cc:1888] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:14.457562 738 tablet_replica.cc:330] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:14.457624 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet b0e8b1909df34fce92f94d6af5f5c3b6 (table rebalance_test_table_12 [id=e2320262ebe84e399b45c01bf9a7d73d]) successfully deleted
I20240502 20:46:14.457701 738 raft_consensus.cc:2229] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:14.457876 738 raft_consensus.cc:2258] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:14.458374 738 ts_tablet_manager.cc:1875] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:14.459914 738 ts_tablet_manager.cc:1888] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:14.459980 738 log.cc:1196] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/f65eb8cde25e483b9c790740ccfd357c
I20240502 20:46:14.460160 738 ts_tablet_manager.cc:1909] T f65eb8cde25e483b9c790740ccfd357c P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:14.460389 739 tablet_replica.cc:330] T 94bdf135dd3a47d39e7b76dbfd524386 P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:14.460523 739 raft_consensus.cc:2229] T 94bdf135dd3a47d39e7b76dbfd524386 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:14.460630 739 raft_consensus.cc:2258] T 94bdf135dd3a47d39e7b76dbfd524386 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:14.460846 32088 log.cc:1196] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/94bdf135dd3a47d39e7b76dbfd524386
I20240502 20:46:14.461141 32088 ts_tablet_manager.cc:1909] T 94bdf135dd3a47d39e7b76dbfd524386 P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:14.461205 739 ts_tablet_manager.cc:1875] T 94bdf135dd3a47d39e7b76dbfd524386 P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:14.461808 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet 94bdf135dd3a47d39e7b76dbfd524386 (table rebalance_test_table_12 [id=e2320262ebe84e399b45c01bf9a7d73d]) successfully deleted
I20240502 20:46:14.463014 738 tablet_replica.cc:330] T 94bdf135dd3a47d39e7b76dbfd524386 P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:14.463133 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet f65eb8cde25e483b9c790740ccfd357c (table rebalance_test_table_12 [id=e2320262ebe84e399b45c01bf9a7d73d]) successfully deleted
I20240502 20:46:14.464151 739 ts_tablet_manager.cc:1888] T 94bdf135dd3a47d39e7b76dbfd524386 P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:14.464215 739 log.cc:1196] T 94bdf135dd3a47d39e7b76dbfd524386 P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/94bdf135dd3a47d39e7b76dbfd524386
I20240502 20:46:14.464432 739 ts_tablet_manager.cc:1909] T 94bdf135dd3a47d39e7b76dbfd524386 P e6b60803d0764d228c863e2da65e3691: Deleting consensus metadata
I20240502 20:46:14.464897 738 raft_consensus.cc:2229] T 94bdf135dd3a47d39e7b76dbfd524386 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:14.465039 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet 94bdf135dd3a47d39e7b76dbfd524386 (table rebalance_test_table_12 [id=e2320262ebe84e399b45c01bf9a7d73d]) successfully deleted
I20240502 20:46:14.465101 738 raft_consensus.cc:2258] T 94bdf135dd3a47d39e7b76dbfd524386 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:14.465507 738 ts_tablet_manager.cc:1875] T 94bdf135dd3a47d39e7b76dbfd524386 P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:14.467300 738 ts_tablet_manager.cc:1888] T 94bdf135dd3a47d39e7b76dbfd524386 P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:14.467383 738 log.cc:1196] T 94bdf135dd3a47d39e7b76dbfd524386 P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/94bdf135dd3a47d39e7b76dbfd524386
I20240502 20:46:14.467693 738 ts_tablet_manager.cc:1909] T 94bdf135dd3a47d39e7b76dbfd524386 P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:14.468298 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet 94bdf135dd3a47d39e7b76dbfd524386 (table rebalance_test_table_12 [id=e2320262ebe84e399b45c01bf9a7d73d]) successfully deleted
I20240502 20:46:14.488533 29175 consensus_queue.cc:237] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:14.489781 29315 raft_consensus.cc:1261] T d167293496fb4334ae670a62a2f80fa9 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:14.489784 31494 raft_consensus.cc:1261] T d167293496fb4334ae670a62a2f80fa9 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:14.490150 32213 consensus_queue.cc:1035] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:14.490352 642 consensus_queue.cc:1035] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:14.492293 32213 raft_consensus.cc:2932] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:14.492426 29036 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:14.492554 29036 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:14.493770 31494 raft_consensus.cc:2932] T d167293496fb4334ae670a62a2f80fa9 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:14.494995 29315 raft_consensus.cc:2932] T d167293496fb4334ae670a62a2f80fa9 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:14.495769 29036 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 2a03d782933b4c8bb95a97c142b22388
I20240502 20:46:14.496479 31737 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a03d782933b4c8bb95a97c142b22388 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:14 UTC) from {username='slave'} at 127.0.0.1:33864
I20240502 20:46:14.496762 759 tablet_replica.cc:330] T 2a03d782933b4c8bb95a97c142b22388 P e7ab6b9053664ee5aa71c7978e26fe6a: stopping tablet replica
I20240502 20:46:14.496897 759 raft_consensus.cc:2229] T 2a03d782933b4c8bb95a97c142b22388 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:14.497016 759 raft_consensus.cc:2258] T 2a03d782933b4c8bb95a97c142b22388 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:14.496981 29037 catalog_manager.cc:5484] T d167293496fb4334ae670a62a2f80fa9 P 7796ea3fba294d17a34386873969431d reported cstate change: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:14.497769 759 ts_tablet_manager.cc:1875] T 2a03d782933b4c8bb95a97c142b22388 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:14.498667 31606 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a03d782933b4c8bb95a97c142b22388 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:14 UTC) from {username='slave'} at 127.0.0.1:58346
I20240502 20:46:14.498858 508 tablet_replica.cc:330] T 2a03d782933b4c8bb95a97c142b22388 P 315915efcf2b413ca7f9d874338e1b2d: stopping tablet replica
I20240502 20:46:14.498989 508 raft_consensus.cc:2229] T 2a03d782933b4c8bb95a97c142b22388 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:14.499107 508 raft_consensus.cc:2258] T 2a03d782933b4c8bb95a97c142b22388 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:14.499616 31474 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a03d782933b4c8bb95a97c142b22388 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:14 UTC) from {username='slave'} at 127.0.0.1:40276
I20240502 20:46:14.499821 508 ts_tablet_manager.cc:1875] T 2a03d782933b4c8bb95a97c142b22388 P 315915efcf2b413ca7f9d874338e1b2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:14.499915 760 tablet_replica.cc:330] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe: stopping tablet replica
I20240502 20:46:14.500043 760 raft_consensus.cc:2229] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:14.500396 760 raft_consensus.cc:2258] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:14.501065 760 ts_tablet_manager.cc:1875] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:14.501816 759 ts_tablet_manager.cc:1888] T 2a03d782933b4c8bb95a97c142b22388 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:14.501850 508 ts_tablet_manager.cc:1888] T 2a03d782933b4c8bb95a97c142b22388 P 315915efcf2b413ca7f9d874338e1b2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:14.501880 759 log.cc:1196] T 2a03d782933b4c8bb95a97c142b22388 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/wals/2a03d782933b4c8bb95a97c142b22388
I20240502 20:46:14.501916 508 log.cc:1196] T 2a03d782933b4c8bb95a97c142b22388 P 315915efcf2b413ca7f9d874338e1b2d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal/wals/2a03d782933b4c8bb95a97c142b22388
I20240502 20:46:14.502130 759 ts_tablet_manager.cc:1909] T 2a03d782933b4c8bb95a97c142b22388 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting consensus metadata
I20240502 20:46:14.502171 508 ts_tablet_manager.cc:1909] T 2a03d782933b4c8bb95a97c142b22388 P 315915efcf2b413ca7f9d874338e1b2d: Deleting consensus metadata
I20240502 20:46:14.502611 29426 tablet_service.cc:1514] Processing DeleteTablet for tablet d167293496fb4334ae670a62a2f80fa9 with delete_type TABLET_DATA_TOMBSTONED (TS dbcdc6ac635b44b185c3eba63f66e9d6 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:14.502867 445 tablet_replica.cc:330] T d167293496fb4334ae670a62a2f80fa9 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:14.502864 29026 catalog_manager.cc:4830] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): tablet 2a03d782933b4c8bb95a97c142b22388 (table rebalancer_dynamic_table_new_name [id=d409c40457214940a8c89c7071ba7749]) successfully deleted
I20240502 20:46:14.503010 445 raft_consensus.cc:2229] T d167293496fb4334ae670a62a2f80fa9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:14.503202 445 raft_consensus.cc:2258] T d167293496fb4334ae670a62a2f80fa9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:14.503726 29027 catalog_manager.cc:4830] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): tablet 2a03d782933b4c8bb95a97c142b22388 (table rebalancer_dynamic_table_new_name [id=d409c40457214940a8c89c7071ba7749]) successfully deleted
I20240502 20:46:14.503746 445 ts_tablet_manager.cc:1875] T d167293496fb4334ae670a62a2f80fa9 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:14.503924 760 ts_tablet_manager.cc:1888] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:14.504034 760 log.cc:1196] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/wals/2a03d782933b4c8bb95a97c142b22388
I20240502 20:46:14.504421 760 ts_tablet_manager.cc:1909] T 2a03d782933b4c8bb95a97c142b22388 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting consensus metadata
I20240502 20:46:14.505199 29028 catalog_manager.cc:4830] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): tablet 2a03d782933b4c8bb95a97c142b22388 (table rebalancer_dynamic_table_new_name [id=d409c40457214940a8c89c7071ba7749]) successfully deleted
I20240502 20:46:14.505754 445 ts_tablet_manager.cc:1888] T d167293496fb4334ae670a62a2f80fa9 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 20:46:14.505821 445 log.cc:1196] T d167293496fb4334ae670a62a2f80fa9 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/d167293496fb4334ae670a62a2f80fa9
I20240502 20:46:14.506291 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet d167293496fb4334ae670a62a2f80fa9 (table rebalance_test_table_11 [id=cc807ad977d146dc84be7ee2897657fc]) successfully deleted
I20240502 20:46:14.584625 31606 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:46:14.584909 29294 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:14.587378 29162 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:14.589584 31474 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:46:14.600158 29037 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:14.606267 29162 tablet_service.cc:1467] Processing CreateTablet for tablet 87b5d273af614d0ab9e905ed8c9839ff (DEFAULT_TABLE table=rebalancer_dynamic_table [id=bde16a9c17ff4e39ab2994da0d99edf0]), partition=
I20240502 20:46:14.606877 31474 tablet_service.cc:1467] Processing CreateTablet for tablet 87b5d273af614d0ab9e905ed8c9839ff (DEFAULT_TABLE table=rebalancer_dynamic_table [id=bde16a9c17ff4e39ab2994da0d99edf0]), partition=
I20240502 20:46:14.607115 31474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87b5d273af614d0ab9e905ed8c9839ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:14.609208 436 tablet_bootstrap.cc:492] T 87b5d273af614d0ab9e905ed8c9839ff P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:14.609403 31868 tablet_service.cc:1467] Processing CreateTablet for tablet 87b5d273af614d0ab9e905ed8c9839ff (DEFAULT_TABLE table=rebalancer_dynamic_table [id=bde16a9c17ff4e39ab2994da0d99edf0]), partition=
I20240502 20:46:14.609642 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87b5d273af614d0ab9e905ed8c9839ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:14.610075 436 tablet_bootstrap.cc:654] T 87b5d273af614d0ab9e905ed8c9839ff P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:14.610322 31737 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:14.610941 29556 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:14.611475 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87b5d273af614d0ab9e905ed8c9839ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:14.611754 681 tablet_bootstrap.cc:492] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:14.612589 681 tablet_bootstrap.cc:654] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:14.613662 686 tablet_bootstrap.cc:492] T 87b5d273af614d0ab9e905ed8c9839ff P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:14.614495 686 tablet_bootstrap.cc:654] T 87b5d273af614d0ab9e905ed8c9839ff P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:14.615428 31868 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:46:14.615922 686 tablet_bootstrap.cc:492] T 87b5d273af614d0ab9e905ed8c9839ff P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:14.616009 686 ts_tablet_manager.cc:1367] T 87b5d273af614d0ab9e905ed8c9839ff P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:14.616166 436 tablet_bootstrap.cc:492] T 87b5d273af614d0ab9e905ed8c9839ff P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:14.616240 436 ts_tablet_manager.cc:1367] T 87b5d273af614d0ab9e905ed8c9839ff P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 20:46:14.616360 686 raft_consensus.cc:348] T 87b5d273af614d0ab9e905ed8c9839ff P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:14.616438 686 raft_consensus.cc:374] T 87b5d273af614d0ab9e905ed8c9839ff P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:14.616456 686 raft_consensus.cc:729] T 87b5d273af614d0ab9e905ed8c9839ff P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:14.616554 686 consensus_queue.cc:260] T 87b5d273af614d0ab9e905ed8c9839ff P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:14.616588 436 raft_consensus.cc:348] T 87b5d273af614d0ab9e905ed8c9839ff P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:14.616681 436 raft_consensus.cc:374] T 87b5d273af614d0ab9e905ed8c9839ff P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:14.616710 436 raft_consensus.cc:729] T 87b5d273af614d0ab9e905ed8c9839ff P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:14.616786 686 ts_tablet_manager.cc:1398] T 87b5d273af614d0ab9e905ed8c9839ff P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:14.616822 436 consensus_queue.cc:260] T 87b5d273af614d0ab9e905ed8c9839ff P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:14.617103 436 ts_tablet_manager.cc:1398] T 87b5d273af614d0ab9e905ed8c9839ff P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:14.621373 681 tablet_bootstrap.cc:492] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:14.621466 681 ts_tablet_manager.cc:1367] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.002s
I20240502 20:46:14.621955 681 raft_consensus.cc:348] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:14.622056 681 raft_consensus.cc:374] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:14.622079 681 raft_consensus.cc:729] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:14.622195 681 consensus_queue.cc:260] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:14.622975 29426 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:14.622435 681 ts_tablet_manager.cc:1398] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:14.669731 32160 consensus_queue.cc:1035] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:14.673338 32172 raft_consensus.cc:482] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:14.673461 32172 raft_consensus.cc:504] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:14.673693 605 consensus_queue.cc:1035] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:14.673861 32172 leader_election.cc:288] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:14.674361 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87b5d273af614d0ab9e905ed8c9839ff" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:14.674549 31494 raft_consensus.cc:2454] T 87b5d273af614d0ab9e905ed8c9839ff P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:14.675204 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87b5d273af614d0ab9e905ed8c9839ff" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:14.675276 31823 leader_election.cc:302] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:14.675340 29175 raft_consensus.cc:2454] T 87b5d273af614d0ab9e905ed8c9839ff P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:14.675422 32172 raft_consensus.cc:2781] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:14.675462 32172 raft_consensus.cc:482] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:14.675494 32172 raft_consensus.cc:3037] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:14.676471 32172 raft_consensus.cc:504] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:14.676784 32172 leader_election.cc:288] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:14.677095 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87b5d273af614d0ab9e905ed8c9839ff" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:14.677199 29175 raft_consensus.cc:3037] T 87b5d273af614d0ab9e905ed8c9839ff P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:14.677173 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87b5d273af614d0ab9e905ed8c9839ff" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:14.677296 31494 raft_consensus.cc:3037] T 87b5d273af614d0ab9e905ed8c9839ff P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:14.678247 29175 raft_consensus.cc:2454] T 87b5d273af614d0ab9e905ed8c9839ff P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:14.678320 31494 raft_consensus.cc:2454] T 87b5d273af614d0ab9e905ed8c9839ff P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:14.678671 31823 leader_election.cc:302] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:14.678934 32172 raft_consensus.cc:2781] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:14.679121 32172 raft_consensus.cc:686] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Becoming Leader. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Running, Role: LEADER
I20240502 20:46:14.679273 32172 consensus_queue.cc:237] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:14.681182 29040 catalog_manager.cc:5484] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6b60803d0764d228c863e2da65e3691 (127.22.45.72). New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:14.685540 32191 consensus_queue.cc:1035] T f3c220fbd7df4cefb467ad88e99941b8 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:14.689577 830 consensus_queue.cc:1035] T f3c220fbd7df4cefb467ad88e99941b8 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:14.694658 29175 consensus_queue.cc:237] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:14.695809 29315 raft_consensus.cc:1261] T f19fa5a9a0e840c1bc0bc379a1907383 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:14.695850 31757 raft_consensus.cc:1261] T f19fa5a9a0e840c1bc0bc379a1907383 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:14.696199 642 consensus_queue.cc:1035] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:14.696416 32213 consensus_queue.cc:1035] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:14.698074 32213 raft_consensus.cc:2932] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:14.698523 31757 raft_consensus.cc:2932] T f19fa5a9a0e840c1bc0bc379a1907383 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:14.698769 29315 raft_consensus.cc:2932] T f19fa5a9a0e840c1bc0bc379a1907383 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:14.701208 29037 catalog_manager.cc:5484] T f19fa5a9a0e840c1bc0bc379a1907383 P 7796ea3fba294d17a34386873969431d reported cstate change: config changed from index 4 to 5, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) evicted. New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:14.704442 29556 tablet_service.cc:1514] Processing DeleteTablet for tablet f19fa5a9a0e840c1bc0bc379a1907383 with delete_type TABLET_DATA_TOMBSTONED (TS db17f26a73f3447b92aa9a947e3a7057 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:14.704660 32088 tablet_replica.cc:330] T f19fa5a9a0e840c1bc0bc379a1907383 P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:14.704792 32088 raft_consensus.cc:2229] T f19fa5a9a0e840c1bc0bc379a1907383 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:14.704942 32088 raft_consensus.cc:2258] T f19fa5a9a0e840c1bc0bc379a1907383 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:14.705533 32088 ts_tablet_manager.cc:1875] T f19fa5a9a0e840c1bc0bc379a1907383 P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:14.707726 32088 ts_tablet_manager.cc:1888] T f19fa5a9a0e840c1bc0bc379a1907383 P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 20:46:14.707793 32088 log.cc:1196] T f19fa5a9a0e840c1bc0bc379a1907383 P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/f19fa5a9a0e840c1bc0bc379a1907383
I20240502 20:46:14.708981 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet f19fa5a9a0e840c1bc0bc379a1907383 (table rebalance_test_table_17 [id=b1432bc7242c45e38bf75a9c91c6646c]) successfully deleted
I20240502 20:46:14.784173 29556 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:14.784548 29294 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:14.787813 31606 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:46:14.794979 29426 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:14.795225 29042 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:14.795854 29162 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:14.803624 31494 raft_consensus.cc:1261] T 87b5d273af614d0ab9e905ed8c9839ff P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:14.803974 31474 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:46:14.804028 822 consensus_queue.cc:1035] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:14.804927 29175 raft_consensus.cc:1261] T 87b5d273af614d0ab9e905ed8c9839ff P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:14.806820 822 consensus_queue.cc:1035] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:14.811545 29036 catalog_manager.cc:5576] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267) does not have the latest schema for tablet 87b5d273af614d0ab9e905ed8c9839ff (table rebalancer_dynamic_table [id=bde16a9c17ff4e39ab2994da0d99edf0]). Expected version 1 got 0
I20240502 20:46:14.812124 31868 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:14.813345 29042 catalog_manager.cc:5576] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887) does not have the latest schema for tablet 87b5d273af614d0ab9e905ed8c9839ff (table rebalancer_dynamic_table [id=bde16a9c17ff4e39ab2994da0d99edf0]). Expected version 1 got 0
I20240502 20:46:14.816999 29042 catalog_manager.cc:5576] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267) does not have the latest schema for tablet 87b5d273af614d0ab9e905ed8c9839ff (table rebalancer_dynamic_table [id=bde16a9c17ff4e39ab2994da0d99edf0]). Expected version 1 got 0
I20240502 20:46:14.830039 31737 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:14.832221 893 tablet.cc:1722] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:14.833550 891 mvcc.cc:205] Tried to move back new op lower bound from 7023340645592121344 to 7023340645087506432. Current Snapshot: MvccSnapshot[applied={T|T < 7023340645592121344}]
I20240502 20:46:14.833992 893 tablet.cc:1620] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:14.839679 758 consensus_queue.cc:1035] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:14.845502 894 tablet.cc:1722] T 87b5d273af614d0ab9e905ed8c9839ff P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:14.846571 585 mvcc.cc:205] Tried to move back new op lower bound from 7023340645592121344 to 7023340645087506432. Current Snapshot: MvccSnapshot[applied={T|T < 7023340645592121344}]
I20240502 20:46:14.846711 660 tablet.cc:1722] T 87b5d273af614d0ab9e905ed8c9839ff P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:14.846858 894 tablet.cc:1620] T 87b5d273af614d0ab9e905ed8c9839ff P 7796ea3fba294d17a34386873969431d: MemRowSet was empty: no flush needed.
I20240502 20:46:14.847004 660 tablet.cc:1620] T 87b5d273af614d0ab9e905ed8c9839ff P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:14.852699 29042 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=bde16a9c17ff4e39ab2994da0d99edf0] alter complete (version 1)
I20240502 20:46:14.860715 32213 consensus_queue.cc:1035] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:14.892805 29042 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:14.895282 29444 consensus_queue.cc:237] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:14.896734 29574 raft_consensus.cc:1261] T 760d15b2acee4322a26f1f2a965f22cc P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:14.897152 32571 consensus_queue.cc:1035] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:14.897398 31888 raft_consensus.cc:1261] T 760d15b2acee4322a26f1f2a965f22cc P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:14.897827 32571 consensus_queue.cc:1035] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:14.899803 29294 tablet_service.cc:1467] Processing CreateTablet for tablet 0922994ff62d4e648d5132cb08b6a3c8 (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=2005c6c66aec4f4d823774ee2b67a575]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:14.900133 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0922994ff62d4e648d5132cb08b6a3c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:14.900123 29162 tablet_service.cc:1467] Processing CreateTablet for tablet db5e418e67a44480b8d4c302cde3b96c (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=2005c6c66aec4f4d823774ee2b67a575]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:14.900417 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db5e418e67a44480b8d4c302cde3b96c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:14.900328 355 raft_consensus.cc:2932] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:14.902321 31737 tablet_service.cc:1467] Processing CreateTablet for tablet 5d6d5a5b991349a9af564fbc83b20a22 (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=2005c6c66aec4f4d823774ee2b67a575]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:14.902581 686 tablet_bootstrap.cc:492] T db5e418e67a44480b8d4c302cde3b96c P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:14.902616 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d6d5a5b991349a9af564fbc83b20a22. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:14.903384 31868 tablet_service.cc:1467] Processing CreateTablet for tablet 0922994ff62d4e648d5132cb08b6a3c8 (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=2005c6c66aec4f4d823774ee2b67a575]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:14.903537 686 tablet_bootstrap.cc:654] T db5e418e67a44480b8d4c302cde3b96c P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:14.903591 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0922994ff62d4e648d5132cb08b6a3c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:14.904834 686 tablet_bootstrap.cc:492] T db5e418e67a44480b8d4c302cde3b96c P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:14.904919 686 ts_tablet_manager.cc:1367] T db5e418e67a44480b8d4c302cde3b96c P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:14.905241 681 tablet_bootstrap.cc:492] T 0922994ff62d4e648d5132cb08b6a3c8 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:14.905373 686 raft_consensus.cc:348] T db5e418e67a44480b8d4c302cde3b96c P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:14.905411 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 5d6d5a5b991349a9af564fbc83b20a22 (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=2005c6c66aec4f4d823774ee2b67a575]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:14.905476 686 raft_consensus.cc:374] T db5e418e67a44480b8d4c302cde3b96c P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:14.905508 686 raft_consensus.cc:729] T db5e418e67a44480b8d4c302cde3b96c P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:14.905617 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d6d5a5b991349a9af564fbc83b20a22. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:14.905596 29556 tablet_service.cc:1467] Processing CreateTablet for tablet db5e418e67a44480b8d4c302cde3b96c (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=2005c6c66aec4f4d823774ee2b67a575]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:14.905637 686 consensus_queue.cc:260] T db5e418e67a44480b8d4c302cde3b96c P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:14.905865 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db5e418e67a44480b8d4c302cde3b96c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:14.905912 686 ts_tablet_manager.cc:1398] T db5e418e67a44480b8d4c302cde3b96c P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:14.906031 681 tablet_bootstrap.cc:654] T 0922994ff62d4e648d5132cb08b6a3c8 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:14.907572 31606 tablet_service.cc:1467] Processing CreateTablet for tablet db5e418e67a44480b8d4c302cde3b96c (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=2005c6c66aec4f4d823774ee2b67a575]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:14.907786 31606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db5e418e67a44480b8d4c302cde3b96c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:14.908105 920 tablet_bootstrap.cc:492] T db5e418e67a44480b8d4c302cde3b96c P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:14.908949 920 tablet_bootstrap.cc:654] T db5e418e67a44480b8d4c302cde3b96c P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:14.910912 31888 raft_consensus.cc:2932] T 760d15b2acee4322a26f1f2a965f22cc P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:14.911110 29036 catalog_manager.cc:5484] T 760d15b2acee4322a26f1f2a965f22cc P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index 4 to 5, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) evicted. New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:14.911756 31474 tablet_service.cc:1467] Processing CreateTablet for tablet 0922994ff62d4e648d5132cb08b6a3c8 (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=2005c6c66aec4f4d823774ee2b67a575]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:14.912027 31474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0922994ff62d4e648d5132cb08b6a3c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:14.912989 922 tablet_bootstrap.cc:492] T db5e418e67a44480b8d4c302cde3b96c P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:14.913050 31475 tablet_service.cc:1467] Processing CreateTablet for tablet 5d6d5a5b991349a9af564fbc83b20a22 (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=2005c6c66aec4f4d823774ee2b67a575]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:14.913275 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d6d5a5b991349a9af564fbc83b20a22. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:14.913920 922 tablet_bootstrap.cc:654] T db5e418e67a44480b8d4c302cde3b96c P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:14.914292 29574 raft_consensus.cc:2932] T 760d15b2acee4322a26f1f2a965f22cc P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:14.914705 920 tablet_bootstrap.cc:492] T db5e418e67a44480b8d4c302cde3b96c P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:14.914783 920 ts_tablet_manager.cc:1367] T db5e418e67a44480b8d4c302cde3b96c P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 20:46:14.915261 920 raft_consensus.cc:348] T db5e418e67a44480b8d4c302cde3b96c P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:14.915361 920 raft_consensus.cc:374] T db5e418e67a44480b8d4c302cde3b96c P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:14.915393 920 raft_consensus.cc:729] T db5e418e67a44480b8d4c302cde3b96c P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:14.915514 920 consensus_queue.cc:260] T db5e418e67a44480b8d4c302cde3b96c P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:14.916433 920 ts_tablet_manager.cc:1398] T db5e418e67a44480b8d4c302cde3b96c P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:14.918862 436 tablet_bootstrap.cc:492] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:14.919782 436 tablet_bootstrap.cc:654] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:14.920476 918 tablet_bootstrap.cc:492] T 5d6d5a5b991349a9af564fbc83b20a22 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:14.921242 29294 tablet_service.cc:1514] Processing DeleteTablet for tablet 760d15b2acee4322a26f1f2a965f22cc with delete_type TABLET_DATA_TOMBSTONED (TS 09258ac095664fa99caf0707262cde13 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:14.921507 738 tablet_replica.cc:330] T 760d15b2acee4322a26f1f2a965f22cc P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:14.921569 916 tablet_bootstrap.cc:492] T 0922994ff62d4e648d5132cb08b6a3c8 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:14.921643 436 tablet_bootstrap.cc:492] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:14.921658 738 raft_consensus.cc:2229] T 760d15b2acee4322a26f1f2a965f22cc P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:14.921736 436 ts_tablet_manager.cc:1367] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:14.921825 738 raft_consensus.cc:2258] T 760d15b2acee4322a26f1f2a965f22cc P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:14.922185 436 raft_consensus.cc:348] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:14.922286 436 raft_consensus.cc:374] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:14.922317 436 raft_consensus.cc:729] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:14.922327 738 ts_tablet_manager.cc:1875] T 760d15b2acee4322a26f1f2a965f22cc P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:14.922436 436 consensus_queue.cc:260] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:14.922727 436 ts_tablet_manager.cc:1398] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:14.922881 436 tablet_bootstrap.cc:492] T 5d6d5a5b991349a9af564fbc83b20a22 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:14.922902 918 tablet_bootstrap.cc:654] T 5d6d5a5b991349a9af564fbc83b20a22 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:14.923259 916 tablet_bootstrap.cc:654] T 0922994ff62d4e648d5132cb08b6a3c8 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:14.923724 436 tablet_bootstrap.cc:654] T 5d6d5a5b991349a9af564fbc83b20a22 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:14.924321 681 tablet_bootstrap.cc:492] T 0922994ff62d4e648d5132cb08b6a3c8 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:14.924401 681 ts_tablet_manager.cc:1367] T 0922994ff62d4e648d5132cb08b6a3c8 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20240502 20:46:14.924512 738 ts_tablet_manager.cc:1888] T 760d15b2acee4322a26f1f2a965f22cc P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 20:46:14.924575 738 log.cc:1196] T 760d15b2acee4322a26f1f2a965f22cc P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/760d15b2acee4322a26f1f2a965f22cc
I20240502 20:46:14.924813 681 raft_consensus.cc:348] T 0922994ff62d4e648d5132cb08b6a3c8 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:14.924913 681 raft_consensus.cc:374] T 0922994ff62d4e648d5132cb08b6a3c8 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:14.924942 681 raft_consensus.cc:729] T 0922994ff62d4e648d5132cb08b6a3c8 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:14.925060 681 consensus_queue.cc:260] T 0922994ff62d4e648d5132cb08b6a3c8 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:14.925067 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet 760d15b2acee4322a26f1f2a965f22cc (table rebalance_test_table_5 [id=fd2f36435a614aa98d7f33e9646f33a3]) successfully deleted
I20240502 20:46:14.925345 681 ts_tablet_manager.cc:1398] T 0922994ff62d4e648d5132cb08b6a3c8 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:14.925547 681 tablet_bootstrap.cc:492] T 5d6d5a5b991349a9af564fbc83b20a22 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:14.926398 681 tablet_bootstrap.cc:654] T 5d6d5a5b991349a9af564fbc83b20a22 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:14.927968 681 tablet_bootstrap.cc:492] T 5d6d5a5b991349a9af564fbc83b20a22 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:14.928076 916 tablet_bootstrap.cc:492] T 0922994ff62d4e648d5132cb08b6a3c8 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:14.928159 916 ts_tablet_manager.cc:1367] T 0922994ff62d4e648d5132cb08b6a3c8 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 20:46:14.928155 681 ts_tablet_manager.cc:1367] T 5d6d5a5b991349a9af564fbc83b20a22 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:14.928623 916 raft_consensus.cc:348] T 0922994ff62d4e648d5132cb08b6a3c8 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:14.928737 916 raft_consensus.cc:374] T 0922994ff62d4e648d5132cb08b6a3c8 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:14.928767 916 raft_consensus.cc:729] T 0922994ff62d4e648d5132cb08b6a3c8 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:14.928926 916 consensus_queue.cc:260] T 0922994ff62d4e648d5132cb08b6a3c8 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:14.930361 916 ts_tablet_manager.cc:1398] T 0922994ff62d4e648d5132cb08b6a3c8 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:14.930287 681 raft_consensus.cc:348] T 5d6d5a5b991349a9af564fbc83b20a22 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:14.930526 681 raft_consensus.cc:374] T 5d6d5a5b991349a9af564fbc83b20a22 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:14.930616 681 raft_consensus.cc:729] T 5d6d5a5b991349a9af564fbc83b20a22 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:14.930814 681 consensus_queue.cc:260] T 5d6d5a5b991349a9af564fbc83b20a22 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:14.930970 436 tablet_bootstrap.cc:492] T 5d6d5a5b991349a9af564fbc83b20a22 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:14.931066 436 ts_tablet_manager.cc:1367] T 5d6d5a5b991349a9af564fbc83b20a22 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240502 20:46:14.931447 681 ts_tablet_manager.cc:1398] T 5d6d5a5b991349a9af564fbc83b20a22 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:14.931509 436 raft_consensus.cc:348] T 5d6d5a5b991349a9af564fbc83b20a22 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:14.931625 436 raft_consensus.cc:374] T 5d6d5a5b991349a9af564fbc83b20a22 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:14.931664 436 raft_consensus.cc:729] T 5d6d5a5b991349a9af564fbc83b20a22 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:14.931785 436 consensus_queue.cc:260] T 5d6d5a5b991349a9af564fbc83b20a22 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:14.932049 436 ts_tablet_manager.cc:1398] T 5d6d5a5b991349a9af564fbc83b20a22 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:14.933120 922 tablet_bootstrap.cc:492] T db5e418e67a44480b8d4c302cde3b96c P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:14.933229 922 ts_tablet_manager.cc:1367] T db5e418e67a44480b8d4c302cde3b96c P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20240502 20:46:14.933357 918 tablet_bootstrap.cc:492] T 5d6d5a5b991349a9af564fbc83b20a22 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:14.933456 918 ts_tablet_manager.cc:1367] T 5d6d5a5b991349a9af564fbc83b20a22 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 20:46:14.933743 922 raft_consensus.cc:348] T db5e418e67a44480b8d4c302cde3b96c P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:14.933837 922 raft_consensus.cc:374] T db5e418e67a44480b8d4c302cde3b96c P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:14.933873 922 raft_consensus.cc:729] T db5e418e67a44480b8d4c302cde3b96c P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:14.933992 922 consensus_queue.cc:260] T db5e418e67a44480b8d4c302cde3b96c P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:14.933950 918 raft_consensus.cc:348] T 5d6d5a5b991349a9af564fbc83b20a22 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:14.934075 918 raft_consensus.cc:374] T 5d6d5a5b991349a9af564fbc83b20a22 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:14.934114 918 raft_consensus.cc:729] T 5d6d5a5b991349a9af564fbc83b20a22 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:14.934260 922 ts_tablet_manager.cc:1398] T db5e418e67a44480b8d4c302cde3b96c P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:14.934255 918 consensus_queue.cc:260] T 5d6d5a5b991349a9af564fbc83b20a22 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:14.934543 918 ts_tablet_manager.cc:1398] T 5d6d5a5b991349a9af564fbc83b20a22 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:14.935032 32708 raft_consensus.cc:482] T db5e418e67a44480b8d4c302cde3b96c P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:14.935153 32708 raft_consensus.cc:504] T db5e418e67a44480b8d4c302cde3b96c P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:14.935633 32708 leader_election.cc:288] T db5e418e67a44480b8d4c302cde3b96c P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:14.937147 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db5e418e67a44480b8d4c302cde3b96c" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:14.937283 29175 raft_consensus.cc:2454] T db5e418e67a44480b8d4c302cde3b96c P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:14.937536 29513 leader_election.cc:302] T db5e418e67a44480b8d4c302cde3b96c P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:14.937711 32708 raft_consensus.cc:2781] T db5e418e67a44480b8d4c302cde3b96c P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:14.937757 32708 raft_consensus.cc:482] T db5e418e67a44480b8d4c302cde3b96c P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:14.937789 32708 raft_consensus.cc:3037] T db5e418e67a44480b8d4c302cde3b96c P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:14.938875 32708 raft_consensus.cc:504] T db5e418e67a44480b8d4c302cde3b96c P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:14.939220 32708 leader_election.cc:288] T db5e418e67a44480b8d4c302cde3b96c P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:14.939610 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db5e418e67a44480b8d4c302cde3b96c" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:14.939739 29175 raft_consensus.cc:3037] T db5e418e67a44480b8d4c302cde3b96c P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:14.940826 29175 raft_consensus.cc:2454] T db5e418e67a44480b8d4c302cde3b96c P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:14.941264 29513 leader_election.cc:302] T db5e418e67a44480b8d4c302cde3b96c P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:14.941677 32708 raft_consensus.cc:2781] T db5e418e67a44480b8d4c302cde3b96c P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:14.941758 32708 raft_consensus.cc:686] T db5e418e67a44480b8d4c302cde3b96c P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Becoming Leader. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Running, Role: LEADER
I20240502 20:46:14.941890 32708 consensus_queue.cc:237] T db5e418e67a44480b8d4c302cde3b96c P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:14.943871 29036 catalog_manager.cc:5484] T db5e418e67a44480b8d4c302cde3b96c P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: term changed from 0 to 1, leader changed from <none> to db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:14.945123 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db5e418e67a44480b8d4c302cde3b96c" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:14.945186 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db5e418e67a44480b8d4c302cde3b96c" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:14.945258 31626 raft_consensus.cc:2454] T db5e418e67a44480b8d4c302cde3b96c P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:14.952798 29039 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalance_test_table_14" } modify_external_catalogs: true
I20240502 20:46:14.952944 29039 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalance_test_table_14" } modify_external_catalogs: true
I20240502 20:46:14.955315 29039 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet e12d4c6713ca4606a1bc6ef41540dad6
I20240502 20:46:14.955642 29039 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet cd3fb6362a3445208fc3ceec878b9dcf
I20240502 20:46:14.955857 29039 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 37a946abe45e415892ac408cf8489d82
I20240502 20:46:14.956233 29556 tablet_service.cc:1514] Processing DeleteTablet for tablet e12d4c6713ca4606a1bc6ef41540dad6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:14 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:14.956449 32088 tablet_replica.cc:330] T e12d4c6713ca4606a1bc6ef41540dad6 P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:14.956490 29426 tablet_service.cc:1514] Processing DeleteTablet for tablet cd3fb6362a3445208fc3ceec878b9dcf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:14 UTC) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:14.956584 32088 raft_consensus.cc:2229] T e12d4c6713ca4606a1bc6ef41540dad6 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:14.956682 445 tablet_replica.cc:330] T cd3fb6362a3445208fc3ceec878b9dcf P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:14.956722 32088 raft_consensus.cc:2258] T e12d4c6713ca4606a1bc6ef41540dad6 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:14.956805 445 raft_consensus.cc:2229] T cd3fb6362a3445208fc3ceec878b9dcf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:14.956933 445 raft_consensus.cc:2258] T cd3fb6362a3445208fc3ceec878b9dcf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:14.957185 29294 tablet_service.cc:1514] Processing DeleteTablet for tablet e12d4c6713ca4606a1bc6ef41540dad6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:14 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:14.957242 32088 ts_tablet_manager.cc:1875] T e12d4c6713ca4606a1bc6ef41540dad6 P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:14.957435 445 ts_tablet_manager.cc:1875] T cd3fb6362a3445208fc3ceec878b9dcf P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:14.957680 29162 tablet_service.cc:1514] Processing DeleteTablet for tablet e12d4c6713ca4606a1bc6ef41540dad6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:14 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:14.958366 29557 tablet_service.cc:1514] Processing DeleteTablet for tablet 37a946abe45e415892ac408cf8489d82 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:14 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:14.958539 29556 tablet_service.cc:1514] Processing DeleteTablet for tablet cd3fb6362a3445208fc3ceec878b9dcf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:14 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:14.958595 29295 tablet_service.cc:1514] Processing DeleteTablet for tablet cd3fb6362a3445208fc3ceec878b9dcf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:14 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:14.958752 738 tablet_replica.cc:330] T e12d4c6713ca4606a1bc6ef41540dad6 P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:14.958859 738 raft_consensus.cc:2229] T e12d4c6713ca4606a1bc6ef41540dad6 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:14.958972 738 raft_consensus.cc:2258] T e12d4c6713ca4606a1bc6ef41540dad6 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:14.959419 738 ts_tablet_manager.cc:1875] T e12d4c6713ca4606a1bc6ef41540dad6 P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:14.959638 445 ts_tablet_manager.cc:1888] T cd3fb6362a3445208fc3ceec878b9dcf P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:14.959700 445 log.cc:1196] T cd3fb6362a3445208fc3ceec878b9dcf P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/cd3fb6362a3445208fc3ceec878b9dcf
I20240502 20:46:14.959928 445 ts_tablet_manager.cc:1909] T cd3fb6362a3445208fc3ceec878b9dcf P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting consensus metadata
I20240502 20:46:14.960541 29293 tablet_service.cc:1514] Processing DeleteTablet for tablet 37a946abe45e415892ac408cf8489d82 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:14 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:14.960661 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet cd3fb6362a3445208fc3ceec878b9dcf (table rebalance_test_table_14 [id=b209cd0423394c4ea3701bdf9e25d73f]) successfully deleted
I20240502 20:46:14.961232 738 ts_tablet_manager.cc:1888] T e12d4c6713ca4606a1bc6ef41540dad6 P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:14.961309 738 log.cc:1196] T e12d4c6713ca4606a1bc6ef41540dad6 P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/e12d4c6713ca4606a1bc6ef41540dad6
I20240502 20:46:14.961553 738 ts_tablet_manager.cc:1909] T e12d4c6713ca4606a1bc6ef41540dad6 P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:14.962181 738 tablet_replica.cc:330] T cd3fb6362a3445208fc3ceec878b9dcf P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:14.962249 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet e12d4c6713ca4606a1bc6ef41540dad6 (table rebalance_test_table_14 [id=b209cd0423394c4ea3701bdf9e25d73f]) successfully deleted
I20240502 20:46:14.962353 29163 tablet_service.cc:1514] Processing DeleteTablet for tablet 37a946abe45e415892ac408cf8489d82 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:14 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:14.962590 738 raft_consensus.cc:2229] T cd3fb6362a3445208fc3ceec878b9dcf P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:14.962760 738 raft_consensus.cc:2258] T cd3fb6362a3445208fc3ceec878b9dcf P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:14.963238 738 ts_tablet_manager.cc:1875] T cd3fb6362a3445208fc3ceec878b9dcf P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:14.963282 29039 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:14.964190 32088 ts_tablet_manager.cc:1888] T e12d4c6713ca4606a1bc6ef41540dad6 P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:14.964273 32088 log.cc:1196] T e12d4c6713ca4606a1bc6ef41540dad6 P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/e12d4c6713ca4606a1bc6ef41540dad6
I20240502 20:46:14.964510 32088 ts_tablet_manager.cc:1909] T e12d4c6713ca4606a1bc6ef41540dad6 P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:14.965041 32088 tablet_replica.cc:330] T 37a946abe45e415892ac408cf8489d82 P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:14.965170 32088 raft_consensus.cc:2229] T 37a946abe45e415892ac408cf8489d82 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:14.965278 32088 raft_consensus.cc:2258] T 37a946abe45e415892ac408cf8489d82 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:14.965456 738 ts_tablet_manager.cc:1888] T cd3fb6362a3445208fc3ceec878b9dcf P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:14.965515 738 log.cc:1196] T cd3fb6362a3445208fc3ceec878b9dcf P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/cd3fb6362a3445208fc3ceec878b9dcf
I20240502 20:46:14.965695 32088 ts_tablet_manager.cc:1875] T 37a946abe45e415892ac408cf8489d82 P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:14.965747 738 ts_tablet_manager.cc:1909] T cd3fb6362a3445208fc3ceec878b9dcf P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:14.966665 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet e12d4c6713ca4606a1bc6ef41540dad6 (table rebalance_test_table_14 [id=b209cd0423394c4ea3701bdf9e25d73f]) successfully deleted
I20240502 20:46:14.968070 32088 ts_tablet_manager.cc:1888] T 37a946abe45e415892ac408cf8489d82 P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:14.968120 738 tablet_replica.cc:330] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:14.968223 32088 log.cc:1196] T 37a946abe45e415892ac408cf8489d82 P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/37a946abe45e415892ac408cf8489d82
I20240502 20:46:14.968223 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet cd3fb6362a3445208fc3ceec878b9dcf (table rebalance_test_table_14 [id=b209cd0423394c4ea3701bdf9e25d73f]) successfully deleted
I20240502 20:46:14.968297 738 raft_consensus.cc:2229] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:14.968518 32088 ts_tablet_manager.cc:1909] T 37a946abe45e415892ac408cf8489d82 P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:14.968535 738 raft_consensus.cc:2258] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:14.969105 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet 37a946abe45e415892ac408cf8489d82 (table rebalance_test_table_14 [id=b209cd0423394c4ea3701bdf9e25d73f]) successfully deleted
I20240502 20:46:14.969158 32088 tablet_replica.cc:330] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:14.969278 32088 raft_consensus.cc:2229] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:14.969446 32088 raft_consensus.cc:2258] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:14.969844 32088 ts_tablet_manager.cc:1875] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:14.970935 738 ts_tablet_manager.cc:1875] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:14.970994 893 tablet.cc:1722] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:14.971325 893 tablet.cc:1620] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:14.971518 660 tablet.cc:1722] T 87b5d273af614d0ab9e905ed8c9839ff P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:14.971642 32088 ts_tablet_manager.cc:1888] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:14.971709 32088 log.cc:1196] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/cd3fb6362a3445208fc3ceec878b9dcf
I20240502 20:46:14.971807 660 tablet.cc:1620] T 87b5d273af614d0ab9e905ed8c9839ff P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:14.971932 32088 ts_tablet_manager.cc:1909] T cd3fb6362a3445208fc3ceec878b9dcf P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:14.972505 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet cd3fb6362a3445208fc3ceec878b9dcf (table rebalance_test_table_14 [id=b209cd0423394c4ea3701bdf9e25d73f]) successfully deleted
I20240502 20:46:14.972822 943 tablet_replica.cc:330] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:14.972970 943 raft_consensus.cc:2229] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:14.973218 943 raft_consensus.cc:2258] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:14.973742 943 ts_tablet_manager.cc:1875] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:14.974370 894 tablet.cc:1722] T 87b5d273af614d0ab9e905ed8c9839ff P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:14.974684 894 tablet.cc:1620] T 87b5d273af614d0ab9e905ed8c9839ff P 7796ea3fba294d17a34386873969431d: MemRowSet was empty: no flush needed.
I20240502 20:46:14.976495 943 ts_tablet_manager.cc:1888] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:14.976719 943 log.cc:1196] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/e12d4c6713ca4606a1bc6ef41540dad6
I20240502 20:46:14.976948 943 ts_tablet_manager.cc:1909] T e12d4c6713ca4606a1bc6ef41540dad6 P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:14.977643 29039 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=bde16a9c17ff4e39ab2994da0d99edf0] alter complete (version 2)
I20240502 20:46:14.977998 943 tablet_replica.cc:330] T 37a946abe45e415892ac408cf8489d82 P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:14.978133 943 raft_consensus.cc:2229] T 37a946abe45e415892ac408cf8489d82 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:14.978300 943 raft_consensus.cc:2258] T 37a946abe45e415892ac408cf8489d82 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:14.978302 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet e12d4c6713ca4606a1bc6ef41540dad6 (table rebalance_test_table_14 [id=b209cd0423394c4ea3701bdf9e25d73f]) successfully deleted
I20240502 20:46:14.978857 943 ts_tablet_manager.cc:1875] T 37a946abe45e415892ac408cf8489d82 P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:14.979938 738 ts_tablet_manager.cc:1888] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:14.980002 738 log.cc:1196] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/37a946abe45e415892ac408cf8489d82
I20240502 20:46:14.980173 738 ts_tablet_manager.cc:1909] T 37a946abe45e415892ac408cf8489d82 P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:14.980909 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet 37a946abe45e415892ac408cf8489d82 (table rebalance_test_table_14 [id=b209cd0423394c4ea3701bdf9e25d73f]) successfully deleted
I20240502 20:46:14.980964 943 ts_tablet_manager.cc:1888] T 37a946abe45e415892ac408cf8489d82 P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:14.981024 943 log.cc:1196] T 37a946abe45e415892ac408cf8489d82 P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/37a946abe45e415892ac408cf8489d82
I20240502 20:46:14.981241 943 ts_tablet_manager.cc:1909] T 37a946abe45e415892ac408cf8489d82 P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:14.981763 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 37a946abe45e415892ac408cf8489d82 (table rebalance_test_table_14 [id=b209cd0423394c4ea3701bdf9e25d73f]) successfully deleted
I20240502 20:46:15.080466 29034 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:15.086364 893 tablet.cc:1722] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:15.086800 660 tablet.cc:1722] T 87b5d273af614d0ab9e905ed8c9839ff P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:15.089403 894 tablet.cc:1722] T 87b5d273af614d0ab9e905ed8c9839ff P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:15.091325 29035 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=bde16a9c17ff4e39ab2994da0d99edf0] alter complete (version 3)
I20240502 20:46:15.147389 692 raft_consensus.cc:482] T 5d6d5a5b991349a9af564fbc83b20a22 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:15.147521 692 raft_consensus.cc:504] T 5d6d5a5b991349a9af564fbc83b20a22 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:15.148007 692 leader_election.cc:288] T 5d6d5a5b991349a9af564fbc83b20a22 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:15.151906 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d6d5a5b991349a9af564fbc83b20a22" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:15.152077 31888 raft_consensus.cc:2454] T 5d6d5a5b991349a9af564fbc83b20a22 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:15.152441 31691 leader_election.cc:302] T 5d6d5a5b991349a9af564fbc83b20a22 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e6b60803d0764d228c863e2da65e3691, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:15.152596 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d6d5a5b991349a9af564fbc83b20a22" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:15.152757 31494 raft_consensus.cc:2454] T 5d6d5a5b991349a9af564fbc83b20a22 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:15.152853 692 raft_consensus.cc:2781] T 5d6d5a5b991349a9af564fbc83b20a22 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:15.152920 692 raft_consensus.cc:482] T 5d6d5a5b991349a9af564fbc83b20a22 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:15.152962 692 raft_consensus.cc:3037] T 5d6d5a5b991349a9af564fbc83b20a22 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:15.153844 692 raft_consensus.cc:504] T 5d6d5a5b991349a9af564fbc83b20a22 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:15.154167 692 leader_election.cc:288] T 5d6d5a5b991349a9af564fbc83b20a22 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Requested vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:15.154565 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d6d5a5b991349a9af564fbc83b20a22" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:15.154572 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d6d5a5b991349a9af564fbc83b20a22" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:15.154665 31494 raft_consensus.cc:3037] T 5d6d5a5b991349a9af564fbc83b20a22 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:15.154681 31888 raft_consensus.cc:3037] T 5d6d5a5b991349a9af564fbc83b20a22 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:15.155565 31494 raft_consensus.cc:2454] T 5d6d5a5b991349a9af564fbc83b20a22 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:15.155580 31888 raft_consensus.cc:2454] T 5d6d5a5b991349a9af564fbc83b20a22 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:15.155843 31692 leader_election.cc:302] T 5d6d5a5b991349a9af564fbc83b20a22 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:15.156160 692 raft_consensus.cc:2781] T 5d6d5a5b991349a9af564fbc83b20a22 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:15.156332 692 raft_consensus.cc:686] T 5d6d5a5b991349a9af564fbc83b20a22 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Becoming Leader. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Running, Role: LEADER
I20240502 20:46:15.156478 692 consensus_queue.cc:237] T 5d6d5a5b991349a9af564fbc83b20a22 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:15.158505 29034 catalog_manager.cc:5484] T 5d6d5a5b991349a9af564fbc83b20a22 P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: term changed from 0 to 1, leader changed from <none> to e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71). New cstate: current_term: 1 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:15.160248 32108 raft_consensus.cc:482] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:15.160331 32108 raft_consensus.cc:504] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:15.160670 32108 leader_election.cc:288] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:15.161006 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0922994ff62d4e648d5132cb08b6a3c8" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:15.161046 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0922994ff62d4e648d5132cb08b6a3c8" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:15.161154 31888 raft_consensus.cc:2454] T 0922994ff62d4e648d5132cb08b6a3c8 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:15.161154 29315 raft_consensus.cc:2454] T 0922994ff62d4e648d5132cb08b6a3c8 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:15.161428 31429 leader_election.cc:302] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:15.161593 32108 raft_consensus.cc:2781] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:15.161652 32108 raft_consensus.cc:482] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:15.161696 32108 raft_consensus.cc:3037] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:15.162433 32108 raft_consensus.cc:504] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:15.162771 32108 leader_election.cc:288] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:15.163066 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0922994ff62d4e648d5132cb08b6a3c8" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:15.163069 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0922994ff62d4e648d5132cb08b6a3c8" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:15.163167 31888 raft_consensus.cc:3037] T 0922994ff62d4e648d5132cb08b6a3c8 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:15.163178 29315 raft_consensus.cc:3037] T 0922994ff62d4e648d5132cb08b6a3c8 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:15.163864 31888 raft_consensus.cc:2454] T 0922994ff62d4e648d5132cb08b6a3c8 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:15.163924 29315 raft_consensus.cc:2454] T 0922994ff62d4e648d5132cb08b6a3c8 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:15.165098 31430 leader_election.cc:302] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 2a9c1af8cf814030bde56b67ca5c2fbe; no voters:
I20240502 20:46:15.165321 32108 raft_consensus.cc:2781] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:15.165455 32108 raft_consensus.cc:686] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Becoming Leader. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Running, Role: LEADER
I20240502 20:46:15.165609 32108 consensus_queue.cc:237] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:15.167411 29034 catalog_manager.cc:5484] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69). New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:15.196111 29035 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:15.196273 29035 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:15.197404 29293 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:15.200213 29035 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 87b5d273af614d0ab9e905ed8c9839ff
I20240502 20:46:15.200826 29163 tablet_service.cc:1514] Processing DeleteTablet for tablet 87b5d273af614d0ab9e905ed8c9839ff with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:15 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:15.201083 943 tablet_replica.cc:330] T 87b5d273af614d0ab9e905ed8c9839ff P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:15.201234 943 raft_consensus.cc:2229] T 87b5d273af614d0ab9e905ed8c9839ff P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:15.201275 31475 tablet_service.cc:1514] Processing DeleteTablet for tablet 87b5d273af614d0ab9e905ed8c9839ff with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:15 UTC) from {username='slave'} at 127.0.0.1:40276
I20240502 20:46:15.201372 943 raft_consensus.cc:2258] T 87b5d273af614d0ab9e905ed8c9839ff P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:15.201557 31867 tablet_service.cc:1514] Processing DeleteTablet for tablet 87b5d273af614d0ab9e905ed8c9839ff with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:15 UTC) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:15.201874 990 tablet_replica.cc:330] T 87b5d273af614d0ab9e905ed8c9839ff P 2a9c1af8cf814030bde56b67ca5c2fbe: stopping tablet replica
I20240502 20:46:15.202004 990 raft_consensus.cc:2229] T 87b5d273af614d0ab9e905ed8c9839ff P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:15.202172 990 raft_consensus.cc:2258] T 87b5d273af614d0ab9e905ed8c9839ff P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:15.203099 990 ts_tablet_manager.cc:1875] T 87b5d273af614d0ab9e905ed8c9839ff P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:15.203961 31606 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:46:15.205155 943 ts_tablet_manager.cc:1875] T 87b5d273af614d0ab9e905ed8c9839ff P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:15.205811 29556 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:15.206498 990 ts_tablet_manager.cc:1888] T 87b5d273af614d0ab9e905ed8c9839ff P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:15.206580 990 log.cc:1196] T 87b5d273af614d0ab9e905ed8c9839ff P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/wals/87b5d273af614d0ab9e905ed8c9839ff
I20240502 20:46:15.207254 990 ts_tablet_manager.cc:1909] T 87b5d273af614d0ab9e905ed8c9839ff P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting consensus metadata
I20240502 20:46:15.207356 943 ts_tablet_manager.cc:1888] T 87b5d273af614d0ab9e905ed8c9839ff P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:15.207458 943 log.cc:1196] T 87b5d273af614d0ab9e905ed8c9839ff P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/87b5d273af614d0ab9e905ed8c9839ff
I20240502 20:46:15.207777 943 ts_tablet_manager.cc:1909] T 87b5d273af614d0ab9e905ed8c9839ff P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:15.208129 29028 catalog_manager.cc:4830] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): tablet 87b5d273af614d0ab9e905ed8c9839ff (table rebalancer_dynamic_table_new_name [id=bde16a9c17ff4e39ab2994da0d99edf0]) successfully deleted
I20240502 20:46:15.208602 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 87b5d273af614d0ab9e905ed8c9839ff (table rebalancer_dynamic_table_new_name [id=bde16a9c17ff4e39ab2994da0d99edf0]) successfully deleted
I20240502 20:46:15.209683 31475 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:15.210005 991 tablet_replica.cc:330] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:15.210165 991 raft_consensus.cc:2229] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:15.210362 991 raft_consensus.cc:2258] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:15.211058 991 ts_tablet_manager.cc:1875] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:15.212651 31737 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:46:15.212834 31867 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:46:15.213263 991 ts_tablet_manager.cc:1888] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:15.213325 991 log.cc:1196] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/87b5d273af614d0ab9e905ed8c9839ff
I20240502 20:46:15.213585 991 ts_tablet_manager.cc:1909] T 87b5d273af614d0ab9e905ed8c9839ff P e6b60803d0764d228c863e2da65e3691: Deleting consensus metadata
I20240502 20:46:15.214236 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet 87b5d273af614d0ab9e905ed8c9839ff (table rebalancer_dynamic_table_new_name [id=bde16a9c17ff4e39ab2994da0d99edf0]) successfully deleted
I20240502 20:46:15.214708 29163 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:15.239699 29426 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:15.280604 29315 consensus_queue.cc:237] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:15.281911 31494 raft_consensus.cc:1261] T dcbca797b2d14e76b03e379cfe132597 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:15.281937 29574 raft_consensus.cc:1261] T dcbca797b2d14e76b03e379cfe132597 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:15.282739 830 consensus_queue.cc:1035] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:15.283057 830 consensus_queue.cc:1035] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:15.285006 32191 raft_consensus.cc:2932] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:15.285948 29574 raft_consensus.cc:2932] T dcbca797b2d14e76b03e379cfe132597 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:15.286820 31494 raft_consensus.cc:2932] T dcbca797b2d14e76b03e379cfe132597 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:15.288596 29035 catalog_manager.cc:5484] T dcbca797b2d14e76b03e379cfe132597 P 09258ac095664fa99caf0707262cde13 reported cstate change: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:15.291433 29426 tablet_service.cc:1514] Processing DeleteTablet for tablet dcbca797b2d14e76b03e379cfe132597 with delete_type TABLET_DATA_TOMBSTONED (TS dbcdc6ac635b44b185c3eba63f66e9d6 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:15.291651 445 tablet_replica.cc:330] T dcbca797b2d14e76b03e379cfe132597 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:15.291764 445 raft_consensus.cc:2229] T dcbca797b2d14e76b03e379cfe132597 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:15.291882 445 raft_consensus.cc:2258] T dcbca797b2d14e76b03e379cfe132597 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:15.292371 445 ts_tablet_manager.cc:1875] T dcbca797b2d14e76b03e379cfe132597 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:15.294440 445 ts_tablet_manager.cc:1888] T dcbca797b2d14e76b03e379cfe132597 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 20:46:15.294562 445 log.cc:1196] T dcbca797b2d14e76b03e379cfe132597 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/dcbca797b2d14e76b03e379cfe132597
I20240502 20:46:15.295032 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet dcbca797b2d14e76b03e379cfe132597 (table rebalance_test_table_7 [id=cf326efc9cf1428897a47dd98c65f088]) successfully deleted
I20240502 20:46:15.302098 29034 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:15.307433 29556 tablet_service.cc:1467] Processing CreateTablet for tablet 994d5557455c4708b108dba74ba5fc17 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=de8c6ae97d584d2bacde6f7cf5f7d23f]), partition=
I20240502 20:46:15.307433 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 994d5557455c4708b108dba74ba5fc17 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=de8c6ae97d584d2bacde6f7cf5f7d23f]), partition=
I20240502 20:46:15.307703 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 994d5557455c4708b108dba74ba5fc17. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:15.307698 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 994d5557455c4708b108dba74ba5fc17. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:15.309908 29293 tablet_service.cc:1467] Processing CreateTablet for tablet 994d5557455c4708b108dba74ba5fc17 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=de8c6ae97d584d2bacde6f7cf5f7d23f]), partition=
I20240502 20:46:15.310103 29293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 994d5557455c4708b108dba74ba5fc17. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:15.311875 916 tablet_bootstrap.cc:492] T 994d5557455c4708b108dba74ba5fc17 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:15.312793 916 tablet_bootstrap.cc:654] T 994d5557455c4708b108dba74ba5fc17 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:15.313169 681 tablet_bootstrap.cc:492] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:15.313897 916 tablet_bootstrap.cc:492] T 994d5557455c4708b108dba74ba5fc17 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:15.313975 916 ts_tablet_manager.cc:1367] T 994d5557455c4708b108dba74ba5fc17 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:15.314016 681 tablet_bootstrap.cc:654] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:15.314440 916 raft_consensus.cc:348] T 994d5557455c4708b108dba74ba5fc17 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:15.314589 916 raft_consensus.cc:374] T 994d5557455c4708b108dba74ba5fc17 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:15.314622 916 raft_consensus.cc:729] T 994d5557455c4708b108dba74ba5fc17 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:15.314620 920 tablet_bootstrap.cc:492] T 994d5557455c4708b108dba74ba5fc17 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:15.314750 916 consensus_queue.cc:260] T 994d5557455c4708b108dba74ba5fc17 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:15.315029 916 ts_tablet_manager.cc:1398] T 994d5557455c4708b108dba74ba5fc17 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:15.315438 920 tablet_bootstrap.cc:654] T 994d5557455c4708b108dba74ba5fc17 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:15.316439 920 tablet_bootstrap.cc:492] T 994d5557455c4708b108dba74ba5fc17 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:15.316531 920 ts_tablet_manager.cc:1367] T 994d5557455c4708b108dba74ba5fc17 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 20:46:15.316973 920 raft_consensus.cc:348] T 994d5557455c4708b108dba74ba5fc17 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:15.317075 920 raft_consensus.cc:374] T 994d5557455c4708b108dba74ba5fc17 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:15.317104 920 raft_consensus.cc:729] T 994d5557455c4708b108dba74ba5fc17 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:15.317240 920 consensus_queue.cc:260] T 994d5557455c4708b108dba74ba5fc17 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:15.317502 920 ts_tablet_manager.cc:1398] T 994d5557455c4708b108dba74ba5fc17 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:15.319118 681 tablet_bootstrap.cc:492] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:15.319208 681 ts_tablet_manager.cc:1367] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20240502 20:46:15.319644 681 raft_consensus.cc:348] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:15.319741 681 raft_consensus.cc:374] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:15.319765 681 raft_consensus.cc:729] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:15.319867 681 consensus_queue.cc:260] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:15.320096 681 ts_tablet_manager.cc:1398] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:15.330684 923 consensus_queue.cc:1035] T db5e418e67a44480b8d4c302cde3b96c P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:15.333657 31625 raft_consensus.cc:3037] T db5e418e67a44480b8d4c302cde3b96c P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:15.338604 32708 consensus_queue.cc:1035] T db5e418e67a44480b8d4c302cde3b96c P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:15.406821 822 raft_consensus.cc:482] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:15.406941 822 raft_consensus.cc:504] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:15.407426 822 leader_election.cc:288] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:15.407744 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "994d5557455c4708b108dba74ba5fc17" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:15.407886 29574 raft_consensus.cc:2454] T 994d5557455c4708b108dba74ba5fc17 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:15.408193 31822 leader_election.cc:302] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:15.408396 822 raft_consensus.cc:2781] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:15.408447 822 raft_consensus.cc:482] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:15.408484 822 raft_consensus.cc:3037] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:15.409299 822 raft_consensus.cc:504] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:15.409686 822 leader_election.cc:288] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:15.409942 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "994d5557455c4708b108dba74ba5fc17" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:15.410049 29574 raft_consensus.cc:3037] T 994d5557455c4708b108dba74ba5fc17 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:15.411049 29574 raft_consensus.cc:2454] T 994d5557455c4708b108dba74ba5fc17 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:15.411326 31822 leader_election.cc:302] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:15.411496 822 raft_consensus.cc:2781] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:15.411578 822 raft_consensus.cc:686] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Becoming Leader. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Running, Role: LEADER
I20240502 20:46:15.411701 822 consensus_queue.cc:237] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:15.412968 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "994d5557455c4708b108dba74ba5fc17" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:15.412987 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "994d5557455c4708b108dba74ba5fc17" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:15.413098 29315 raft_consensus.cc:2454] T 994d5557455c4708b108dba74ba5fc17 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:15.413638 29037 catalog_manager.cc:5484] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6b60803d0764d228c863e2da65e3691 (127.22.45.72). New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:15.432139 29556 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:15.432674 31475 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:15.440193 29293 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:15.440426 31606 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:46:15.441099 29163 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:15.445277 31737 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:46:15.455739 31867 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:15.463771 29037 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalance_test_table_16" } modify_external_catalogs: true
I20240502 20:46:15.463934 29037 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalance_test_table_16" } modify_external_catalogs: true
I20240502 20:46:15.466691 29037 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 4 replicas of tablet e01c5b97e2d14976812691b78a5143c5
I20240502 20:46:15.467060 29037 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 1d0d00056f794ed89631137c89569385
I20240502 20:46:15.467233 29037 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 791a0a29d67447fcb54e38e848680330
I20240502 20:46:15.468011 29163 tablet_service.cc:1514] Processing DeleteTablet for tablet e01c5b97e2d14976812691b78a5143c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:15 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:15.468112 29556 tablet_service.cc:1514] Processing DeleteTablet for tablet e01c5b97e2d14976812691b78a5143c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:15 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:15.468216 29162 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d0d00056f794ed89631137c89569385 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:15 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:15.468286 32088 tablet_replica.cc:330] T e01c5b97e2d14976812691b78a5143c5 P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:15.468344 29557 tablet_service.cc:1514] Processing DeleteTablet for tablet 791a0a29d67447fcb54e38e848680330 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:15 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:15.468417 32088 raft_consensus.cc:2229] T e01c5b97e2d14976812691b78a5143c5 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:15.468472 943 tablet_replica.cc:330] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:15.468576 32088 raft_consensus.cc:2258] T e01c5b97e2d14976812691b78a5143c5 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:15.468591 943 raft_consensus.cc:2229] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:15.468715 29294 tablet_service.cc:1514] Processing DeleteTablet for tablet 791a0a29d67447fcb54e38e848680330 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:15 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:15.468837 943 raft_consensus.cc:2258] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:15.469138 32088 ts_tablet_manager.cc:1875] T e01c5b97e2d14976812691b78a5143c5 P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:15.469128 29426 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d0d00056f794ed89631137c89569385 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:15 UTC) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:15.469264 29424 tablet_service.cc:1514] Processing DeleteTablet for tablet 791a0a29d67447fcb54e38e848680330 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:15 UTC) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:15.469326 943 ts_tablet_manager.cc:1875] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:15.469368 445 tablet_replica.cc:330] T 1d0d00056f794ed89631137c89569385 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:15.469486 445 raft_consensus.cc:2229] T 1d0d00056f794ed89631137c89569385 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:15.469623 445 raft_consensus.cc:2258] T 1d0d00056f794ed89631137c89569385 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:15.470111 445 ts_tablet_manager.cc:1875] T 1d0d00056f794ed89631137c89569385 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:15.470563 29295 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d0d00056f794ed89631137c89569385 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:15 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:15.470635 738 tablet_replica.cc:330] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:15.470767 738 raft_consensus.cc:2229] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:15.470984 738 raft_consensus.cc:2258] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:15.471519 738 ts_tablet_manager.cc:1875] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:15.471627 32088 ts_tablet_manager.cc:1888] T e01c5b97e2d14976812691b78a5143c5 P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:15.471697 32088 log.cc:1196] T e01c5b97e2d14976812691b78a5143c5 P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/e01c5b97e2d14976812691b78a5143c5
I20240502 20:46:15.471927 32088 ts_tablet_manager.cc:1909] T e01c5b97e2d14976812691b78a5143c5 P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:15.472169 445 ts_tablet_manager.cc:1888] T 1d0d00056f794ed89631137c89569385 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:15.472229 445 log.cc:1196] T 1d0d00056f794ed89631137c89569385 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/1d0d00056f794ed89631137c89569385
I20240502 20:46:15.472477 445 ts_tablet_manager.cc:1909] T 1d0d00056f794ed89631137c89569385 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting consensus metadata
I20240502 20:46:15.472599 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet e01c5b97e2d14976812691b78a5143c5 (table rebalance_test_table_16 [id=c5c351da1d6f466a88f1fcf68cd0a72d]) successfully deleted
I20240502 20:46:15.472645 32088 tablet_replica.cc:330] T 791a0a29d67447fcb54e38e848680330 P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:15.472775 32088 raft_consensus.cc:2229] T 791a0a29d67447fcb54e38e848680330 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:15.472895 32088 raft_consensus.cc:2258] T 791a0a29d67447fcb54e38e848680330 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:15.473322 32088 ts_tablet_manager.cc:1875] T 791a0a29d67447fcb54e38e848680330 P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:15.473732 738 ts_tablet_manager.cc:1888] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:15.473789 738 log.cc:1196] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/791a0a29d67447fcb54e38e848680330
I20240502 20:46:15.474022 738 ts_tablet_manager.cc:1909] T 791a0a29d67447fcb54e38e848680330 P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:15.470563 29293 tablet_service.cc:1514] Processing DeleteTablet for tablet e01c5b97e2d14976812691b78a5143c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:15 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:15.474869 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet 1d0d00056f794ed89631137c89569385 (table rebalance_test_table_16 [id=c5c351da1d6f466a88f1fcf68cd0a72d]) successfully deleted
I20240502 20:46:15.475124 29424 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:15.475328 31475 tablet_service.cc:1514] Processing DeleteTablet for tablet e01c5b97e2d14976812691b78a5143c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:15 UTC) from {username='slave'} at 127.0.0.1:40276
I20240502 20:46:15.475500 445 tablet_replica.cc:330] T 791a0a29d67447fcb54e38e848680330 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:15.475530 990 tablet_replica.cc:330] T e01c5b97e2d14976812691b78a5143c5 P 2a9c1af8cf814030bde56b67ca5c2fbe: stopping tablet replica
I20240502 20:46:15.475647 990 raft_consensus.cc:2229] T e01c5b97e2d14976812691b78a5143c5 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:15.475675 445 raft_consensus.cc:2229] T 791a0a29d67447fcb54e38e848680330 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:15.475740 990 raft_consensus.cc:2258] T e01c5b97e2d14976812691b78a5143c5 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:15.476274 990 ts_tablet_manager.cc:1875] T e01c5b97e2d14976812691b78a5143c5 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:15.476780 445 raft_consensus.cc:2258] T 791a0a29d67447fcb54e38e848680330 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:15.477295 445 ts_tablet_manager.cc:1875] T 791a0a29d67447fcb54e38e848680330 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:15.478217 990 ts_tablet_manager.cc:1888] T e01c5b97e2d14976812691b78a5143c5 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:15.478281 990 log.cc:1196] T e01c5b97e2d14976812691b78a5143c5 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/wals/e01c5b97e2d14976812691b78a5143c5
I20240502 20:46:15.480495 445 ts_tablet_manager.cc:1888] T 791a0a29d67447fcb54e38e848680330 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:15.480562 445 log.cc:1196] T 791a0a29d67447fcb54e38e848680330 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/791a0a29d67447fcb54e38e848680330
I20240502 20:46:15.481791 32088 ts_tablet_manager.cc:1888] T 791a0a29d67447fcb54e38e848680330 P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:15.481863 32088 log.cc:1196] T 791a0a29d67447fcb54e38e848680330 P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/791a0a29d67447fcb54e38e848680330
I20240502 20:46:15.481984 445 ts_tablet_manager.cc:1909] T 791a0a29d67447fcb54e38e848680330 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting consensus metadata
I20240502 20:46:15.482095 32088 ts_tablet_manager.cc:1909] T 791a0a29d67447fcb54e38e848680330 P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:15.482403 943 ts_tablet_manager.cc:1888] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:15.482676 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet 791a0a29d67447fcb54e38e848680330 (table rebalance_test_table_16 [id=c5c351da1d6f466a88f1fcf68cd0a72d]) successfully deleted
I20240502 20:46:15.483781 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet 791a0a29d67447fcb54e38e848680330 (table rebalance_test_table_16 [id=c5c351da1d6f466a88f1fcf68cd0a72d]) successfully deleted
I20240502 20:46:15.483870 990 ts_tablet_manager.cc:1909] T e01c5b97e2d14976812691b78a5143c5 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting consensus metadata
I20240502 20:46:15.484747 943 log.cc:1196] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/e01c5b97e2d14976812691b78a5143c5
I20240502 20:46:15.484757 29028 catalog_manager.cc:4830] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): tablet e01c5b97e2d14976812691b78a5143c5 (table rebalance_test_table_16 [id=c5c351da1d6f466a88f1fcf68cd0a72d]) successfully deleted
I20240502 20:46:15.485069 943 ts_tablet_manager.cc:1909] T e01c5b97e2d14976812691b78a5143c5 P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:15.486160 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet 791a0a29d67447fcb54e38e848680330 (table rebalance_test_table_16 [id=c5c351da1d6f466a88f1fcf68cd0a72d]) successfully deleted
I20240502 20:46:15.486420 738 tablet_replica.cc:330] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:15.486660 738 raft_consensus.cc:2229] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:15.486919 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet e01c5b97e2d14976812691b78a5143c5 (table rebalance_test_table_16 [id=c5c351da1d6f466a88f1fcf68cd0a72d]) successfully deleted
I20240502 20:46:15.487237 943 tablet_replica.cc:330] T 1d0d00056f794ed89631137c89569385 P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:15.487413 943 raft_consensus.cc:2229] T 1d0d00056f794ed89631137c89569385 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:15.487545 738 raft_consensus.cc:2258] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:15.487712 943 raft_consensus.cc:2258] T 1d0d00056f794ed89631137c89569385 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:15.488253 738 ts_tablet_manager.cc:1875] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:15.488785 943 ts_tablet_manager.cc:1875] T 1d0d00056f794ed89631137c89569385 P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:15.490967 943 ts_tablet_manager.cc:1888] T 1d0d00056f794ed89631137c89569385 P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:15.491093 943 log.cc:1196] T 1d0d00056f794ed89631137c89569385 P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/1d0d00056f794ed89631137c89569385
I20240502 20:46:15.491386 943 ts_tablet_manager.cc:1909] T 1d0d00056f794ed89631137c89569385 P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:15.492063 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 1d0d00056f794ed89631137c89569385 (table rebalance_test_table_16 [id=c5c351da1d6f466a88f1fcf68cd0a72d]) successfully deleted
I20240502 20:46:15.500106 738 ts_tablet_manager.cc:1888] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:15.500182 738 log.cc:1196] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/1d0d00056f794ed89631137c89569385
I20240502 20:46:15.500389 738 ts_tablet_manager.cc:1909] T 1d0d00056f794ed89631137c89569385 P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:15.500901 738 tablet_replica.cc:330] T e01c5b97e2d14976812691b78a5143c5 P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:15.501034 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet 1d0d00056f794ed89631137c89569385 (table rebalance_test_table_16 [id=c5c351da1d6f466a88f1fcf68cd0a72d]) successfully deleted
I20240502 20:46:15.501256 738 raft_consensus.cc:2229] T e01c5b97e2d14976812691b78a5143c5 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:15.501407 738 raft_consensus.cc:2258] T e01c5b97e2d14976812691b78a5143c5 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:15.501856 738 ts_tablet_manager.cc:1875] T e01c5b97e2d14976812691b78a5143c5 P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:15.503516 738 ts_tablet_manager.cc:1888] T e01c5b97e2d14976812691b78a5143c5 P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:15.503582 738 log.cc:1196] T e01c5b97e2d14976812691b78a5143c5 P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/e01c5b97e2d14976812691b78a5143c5
I20240502 20:46:15.503803 738 ts_tablet_manager.cc:1909] T e01c5b97e2d14976812691b78a5143c5 P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:15.504407 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet e01c5b97e2d14976812691b78a5143c5 (table rebalance_test_table_16 [id=c5c351da1d6f466a88f1fcf68cd0a72d]) successfully deleted
I20240502 20:46:15.522608 29037 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:15.524029 29446 consensus_queue.cc:237] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:15.525537 31757 raft_consensus.cc:1261] T a8916d4762844444bae6fa6f930abbfd P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:15.526983 29315 raft_consensus.cc:3037] T 994d5557455c4708b108dba74ba5fc17 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:15.527942 29315 raft_consensus.cc:1261] T 994d5557455c4708b108dba74ba5fc17 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:15.528309 29574 raft_consensus.cc:1261] T 994d5557455c4708b108dba74ba5fc17 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:15.528810 915 consensus_queue.cc:1035] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:15.528821 32172 consensus_queue.cc:1035] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:15.529120 32172 consensus_queue.cc:1035] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:15.531029 29175 raft_consensus.cc:1261] T a8916d4762844444bae6fa6f930abbfd P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:15.532058 893 tablet.cc:1722] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:15.532188 915 consensus_queue.cc:1035] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:15.532409 893 tablet.cc:1620] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:15.532945 915 raft_consensus.cc:2932] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:15.533267 29037 catalog_manager.cc:5576] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945) does not have the latest schema for tablet 994d5557455c4708b108dba74ba5fc17 (table rebalancer_dynamic_table [id=de8c6ae97d584d2bacde6f7cf5f7d23f]). Expected version 1 got 0
I20240502 20:46:15.534972 31757 raft_consensus.cc:2932] T a8916d4762844444bae6fa6f930abbfd P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:15.536365 29175 raft_consensus.cc:2932] T a8916d4762844444bae6fa6f930abbfd P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:15.538661 29040 catalog_manager.cc:5484] T a8916d4762844444bae6fa6f930abbfd P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index 4 to 5, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) evicted. New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:15.539973 29037 catalog_manager.cc:5576] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897) does not have the latest schema for tablet 994d5557455c4708b108dba74ba5fc17 (table rebalancer_dynamic_table [id=de8c6ae97d584d2bacde6f7cf5f7d23f]). Expected version 1 got 0
I20240502 20:46:15.543289 29036 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=de8c6ae97d584d2bacde6f7cf5f7d23f] alter complete (version 1)
I20240502 20:46:15.544245 1092 tablet.cc:1722] T 994d5557455c4708b108dba74ba5fc17 P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:15.544309 29039 catalog_manager.cc:5576] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945) does not have the latest schema for tablet 994d5557455c4708b108dba74ba5fc17 (table rebalancer_dynamic_table [id=de8c6ae97d584d2bacde6f7cf5f7d23f]). Expected version 1 got 0
I20240502 20:46:15.545559 1092 tablet.cc:1620] T 994d5557455c4708b108dba74ba5fc17 P 09258ac095664fa99caf0707262cde13: MemRowSet was empty: no flush needed.
I20240502 20:46:15.545776 29557 tablet_service.cc:1514] Processing DeleteTablet for tablet a8916d4762844444bae6fa6f930abbfd with delete_type TABLET_DATA_TOMBSTONED (TS db17f26a73f3447b92aa9a947e3a7057 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:15.546000 32088 tablet_replica.cc:330] T a8916d4762844444bae6fa6f930abbfd P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:15.546135 32088 raft_consensus.cc:2229] T a8916d4762844444bae6fa6f930abbfd P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:15.546298 32088 raft_consensus.cc:2258] T a8916d4762844444bae6fa6f930abbfd P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:15.546788 32088 ts_tablet_manager.cc:1875] T a8916d4762844444bae6fa6f930abbfd P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:15.547819 29042 catalog_manager.cc:5576] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897) does not have the latest schema for tablet 994d5557455c4708b108dba74ba5fc17 (table rebalancer_dynamic_table [id=de8c6ae97d584d2bacde6f7cf5f7d23f]). Expected version 1 got 0
I20240502 20:46:15.548530 32088 ts_tablet_manager.cc:1888] T a8916d4762844444bae6fa6f930abbfd P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 20:46:15.548578 32088 log.cc:1196] T a8916d4762844444bae6fa6f930abbfd P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/a8916d4762844444bae6fa6f930abbfd
I20240502 20:46:15.549054 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet a8916d4762844444bae6fa6f930abbfd (table rebalance_test_table_7 [id=cf326efc9cf1428897a47dd98c65f088]) successfully deleted
I20240502 20:46:15.548980 1099 tablet.cc:1722] T 994d5557455c4708b108dba74ba5fc17 P db17f26a73f3447b92aa9a947e3a7057: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:15.549995 1099 tablet.cc:1620] T 994d5557455c4708b108dba74ba5fc17 P db17f26a73f3447b92aa9a947e3a7057: MemRowSet was empty: no flush needed.
I20240502 20:46:15.596098 692 consensus_queue.cc:1035] T 5d6d5a5b991349a9af564fbc83b20a22 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:15.602620 975 consensus_queue.cc:1035] T 5d6d5a5b991349a9af564fbc83b20a22 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:15.609042 977 consensus_queue.cc:1035] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:15.621974 32108 consensus_queue.cc:1035] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:15.635912 29162 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:15.640779 31606 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:46:15.643105 29557 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:15.643882 29293 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:15.646449 31475 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:15.649279 31737 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:46:15.650799 29037 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:15.663797 893 tablet.cc:1722] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:15.664126 893 tablet.cc:1620] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:15.668468 1092 tablet.cc:1722] T 994d5557455c4708b108dba74ba5fc17 P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:15.668809 1092 tablet.cc:1620] T 994d5557455c4708b108dba74ba5fc17 P 09258ac095664fa99caf0707262cde13: MemRowSet was empty: no flush needed.
I20240502 20:46:15.670528 29038 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_10"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:15.670783 29037 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=de8c6ae97d584d2bacde6f7cf5f7d23f] alter complete (version 2)
I20240502 20:46:15.671442 1099 tablet.cc:1722] T 994d5557455c4708b108dba74ba5fc17 P db17f26a73f3447b92aa9a947e3a7057: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:15.671746 1099 tablet.cc:1620] T 994d5557455c4708b108dba74ba5fc17 P db17f26a73f3447b92aa9a947e3a7057: MemRowSet was empty: no flush needed.
I20240502 20:46:15.675789 31867 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:15.676173 29424 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:15.683001 31606 tablet_service.cc:1467] Processing CreateTablet for tablet 211fa6a23bda4c11bb3106080c3ace40 (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=9bcf2b1be3a34dd7a9f3320c43834eda]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:15.683346 31606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 211fa6a23bda4c11bb3106080c3ace40. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:15.683286 31602 tablet_service.cc:1467] Processing CreateTablet for tablet 11ee3029cd744581ba9c5c213ca8fd2b (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=9bcf2b1be3a34dd7a9f3320c43834eda]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:15.683542 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11ee3029cd744581ba9c5c213ca8fd2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:15.683852 29557 tablet_service.cc:1467] Processing CreateTablet for tablet f14cf6deefec46c7adb598f6d7cfd649 (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=9bcf2b1be3a34dd7a9f3320c43834eda]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:15.684103 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f14cf6deefec46c7adb598f6d7cfd649. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:15.684288 31475 tablet_service.cc:1467] Processing CreateTablet for tablet 211fa6a23bda4c11bb3106080c3ace40 (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=9bcf2b1be3a34dd7a9f3320c43834eda]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:15.684542 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 211fa6a23bda4c11bb3106080c3ace40. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:15.686015 31474 tablet_service.cc:1467] Processing CreateTablet for tablet f14cf6deefec46c7adb598f6d7cfd649 (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=9bcf2b1be3a34dd7a9f3320c43834eda]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:15.686241 31474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f14cf6deefec46c7adb598f6d7cfd649. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:15.687105 29424 tablet_service.cc:1467] Processing CreateTablet for tablet 211fa6a23bda4c11bb3106080c3ace40 (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=9bcf2b1be3a34dd7a9f3320c43834eda]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:15.687391 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 211fa6a23bda4c11bb3106080c3ace40. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:15.688952 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 11ee3029cd744581ba9c5c213ca8fd2b (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=9bcf2b1be3a34dd7a9f3320c43834eda]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:15.689220 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11ee3029cd744581ba9c5c213ca8fd2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:15.689563 920 tablet_bootstrap.cc:492] T f14cf6deefec46c7adb598f6d7cfd649 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:15.690389 920 tablet_bootstrap.cc:654] T f14cf6deefec46c7adb598f6d7cfd649 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:15.691962 681 tablet_bootstrap.cc:492] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:15.692871 681 tablet_bootstrap.cc:654] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:15.694291 31736 tablet_service.cc:1467] Processing CreateTablet for tablet f14cf6deefec46c7adb598f6d7cfd649 (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=9bcf2b1be3a34dd7a9f3320c43834eda]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:15.694291 31737 tablet_service.cc:1467] Processing CreateTablet for tablet 11ee3029cd744581ba9c5c213ca8fd2b (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=9bcf2b1be3a34dd7a9f3320c43834eda]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:15.694586 31736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f14cf6deefec46c7adb598f6d7cfd649. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:15.694908 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11ee3029cd744581ba9c5c213ca8fd2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:15.697189 1156 tablet_bootstrap.cc:492] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:15.698110 1156 tablet_bootstrap.cc:654] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:15.699187 1156 tablet_bootstrap.cc:492] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:15.699265 1156 ts_tablet_manager.cc:1367] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 20:46:15.699667 1156 raft_consensus.cc:348] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:15.699780 1156 raft_consensus.cc:374] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:15.699808 1156 raft_consensus.cc:729] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:15.699928 1156 consensus_queue.cc:260] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:15.700192 1156 ts_tablet_manager.cc:1398] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:15.700343 1156 tablet_bootstrap.cc:492] T 11ee3029cd744581ba9c5c213ca8fd2b P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:15.701200 1156 tablet_bootstrap.cc:654] T 11ee3029cd744581ba9c5c213ca8fd2b P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:15.703862 1156 tablet_bootstrap.cc:492] T 11ee3029cd744581ba9c5c213ca8fd2b P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:15.703948 1156 ts_tablet_manager.cc:1367] T 11ee3029cd744581ba9c5c213ca8fd2b P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 20:46:15.704339 1156 raft_consensus.cc:348] T 11ee3029cd744581ba9c5c213ca8fd2b P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:15.704430 1156 raft_consensus.cc:374] T 11ee3029cd744581ba9c5c213ca8fd2b P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:15.704453 1156 raft_consensus.cc:729] T 11ee3029cd744581ba9c5c213ca8fd2b P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:15.704560 1156 consensus_queue.cc:260] T 11ee3029cd744581ba9c5c213ca8fd2b P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:15.704800 1156 ts_tablet_manager.cc:1398] T 11ee3029cd744581ba9c5c213ca8fd2b P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:15.706044 1151 tablet_bootstrap.cc:492] T 11ee3029cd744581ba9c5c213ca8fd2b P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:15.708504 1151 tablet_bootstrap.cc:654] T 11ee3029cd744581ba9c5c213ca8fd2b P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:15.709558 1151 tablet_bootstrap.cc:492] T 11ee3029cd744581ba9c5c213ca8fd2b P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:15.709650 1151 ts_tablet_manager.cc:1367] T 11ee3029cd744581ba9c5c213ca8fd2b P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:15.710032 1151 raft_consensus.cc:348] T 11ee3029cd744581ba9c5c213ca8fd2b P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:15.710130 1151 raft_consensus.cc:374] T 11ee3029cd744581ba9c5c213ca8fd2b P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:15.710160 1151 raft_consensus.cc:729] T 11ee3029cd744581ba9c5c213ca8fd2b P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:15.710371 1151 consensus_queue.cc:260] T 11ee3029cd744581ba9c5c213ca8fd2b P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:15.710654 1151 ts_tablet_manager.cc:1398] T 11ee3029cd744581ba9c5c213ca8fd2b P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:15.710855 1151 tablet_bootstrap.cc:492] T 211fa6a23bda4c11bb3106080c3ace40 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:15.712025 1151 tablet_bootstrap.cc:654] T 211fa6a23bda4c11bb3106080c3ace40 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:15.712888 920 tablet_bootstrap.cc:492] T f14cf6deefec46c7adb598f6d7cfd649 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:15.712988 920 ts_tablet_manager.cc:1367] T f14cf6deefec46c7adb598f6d7cfd649 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.023s user 0.000s sys 0.002s
I20240502 20:46:15.713454 920 raft_consensus.cc:348] T f14cf6deefec46c7adb598f6d7cfd649 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:15.713593 920 raft_consensus.cc:374] T f14cf6deefec46c7adb598f6d7cfd649 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:15.713621 920 raft_consensus.cc:729] T f14cf6deefec46c7adb598f6d7cfd649 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:15.713742 920 consensus_queue.cc:260] T f14cf6deefec46c7adb598f6d7cfd649 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:15.713994 920 ts_tablet_manager.cc:1398] T f14cf6deefec46c7adb598f6d7cfd649 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:15.722883 1154 tablet_bootstrap.cc:492] T 211fa6a23bda4c11bb3106080c3ace40 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:15.723942 1152 tablet_bootstrap.cc:492] T 211fa6a23bda4c11bb3106080c3ace40 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:15.724331 681 tablet_bootstrap.cc:492] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:15.724429 681 ts_tablet_manager.cc:1367] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.033s user 0.001s sys 0.001s
I20240502 20:46:15.724815 1152 tablet_bootstrap.cc:654] T 211fa6a23bda4c11bb3106080c3ace40 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:15.724890 681 raft_consensus.cc:348] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:15.725001 681 raft_consensus.cc:374] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:15.725028 681 raft_consensus.cc:729] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:15.725143 681 consensus_queue.cc:260] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:15.725397 681 ts_tablet_manager.cc:1398] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 20:46:15.727833 1154 tablet_bootstrap.cc:654] T 211fa6a23bda4c11bb3106080c3ace40 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:15.728950 1154 tablet_bootstrap.cc:492] T 211fa6a23bda4c11bb3106080c3ace40 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:15.729095 1154 ts_tablet_manager.cc:1367] T 211fa6a23bda4c11bb3106080c3ace40 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 20:46:15.729576 1154 raft_consensus.cc:348] T 211fa6a23bda4c11bb3106080c3ace40 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:15.729856 1154 raft_consensus.cc:374] T 211fa6a23bda4c11bb3106080c3ace40 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:15.729939 1154 raft_consensus.cc:729] T 211fa6a23bda4c11bb3106080c3ace40 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:15.730782 1152 tablet_bootstrap.cc:492] T 211fa6a23bda4c11bb3106080c3ace40 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:15.730947 1152 ts_tablet_manager.cc:1367] T 211fa6a23bda4c11bb3106080c3ace40 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 20:46:15.730893 1154 consensus_queue.cc:260] T 211fa6a23bda4c11bb3106080c3ace40 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:15.731180 1154 ts_tablet_manager.cc:1398] T 211fa6a23bda4c11bb3106080c3ace40 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:15.731400 1154 tablet_bootstrap.cc:492] T f14cf6deefec46c7adb598f6d7cfd649 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:15.731467 1152 raft_consensus.cc:348] T 211fa6a23bda4c11bb3106080c3ace40 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:15.731683 1152 raft_consensus.cc:374] T 211fa6a23bda4c11bb3106080c3ace40 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:15.731772 1152 raft_consensus.cc:729] T 211fa6a23bda4c11bb3106080c3ace40 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:15.732092 1152 consensus_queue.cc:260] T 211fa6a23bda4c11bb3106080c3ace40 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:15.732493 1152 ts_tablet_manager.cc:1398] T 211fa6a23bda4c11bb3106080c3ace40 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:15.732796 1154 tablet_bootstrap.cc:654] T f14cf6deefec46c7adb598f6d7cfd649 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:15.736011 1154 tablet_bootstrap.cc:492] T f14cf6deefec46c7adb598f6d7cfd649 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:15.736145 1154 ts_tablet_manager.cc:1367] T f14cf6deefec46c7adb598f6d7cfd649 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 20:46:15.736879 1154 raft_consensus.cc:348] T f14cf6deefec46c7adb598f6d7cfd649 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:15.737025 1151 tablet_bootstrap.cc:492] T 211fa6a23bda4c11bb3106080c3ace40 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:15.737061 1154 raft_consensus.cc:374] T f14cf6deefec46c7adb598f6d7cfd649 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:15.737140 1151 ts_tablet_manager.cc:1367] T 211fa6a23bda4c11bb3106080c3ace40 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.000s
I20240502 20:46:15.737181 1154 raft_consensus.cc:729] T f14cf6deefec46c7adb598f6d7cfd649 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:15.737557 1154 consensus_queue.cc:260] T f14cf6deefec46c7adb598f6d7cfd649 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:15.737624 1151 raft_consensus.cc:348] T 211fa6a23bda4c11bb3106080c3ace40 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:15.737747 1151 raft_consensus.cc:374] T 211fa6a23bda4c11bb3106080c3ace40 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:15.737790 1151 raft_consensus.cc:729] T 211fa6a23bda4c11bb3106080c3ace40 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:15.737910 1151 consensus_queue.cc:260] T 211fa6a23bda4c11bb3106080c3ace40 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:15.738175 1151 ts_tablet_manager.cc:1398] T 211fa6a23bda4c11bb3106080c3ace40 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:15.738564 1154 ts_tablet_manager.cc:1398] T f14cf6deefec46c7adb598f6d7cfd649 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:15.738962 32108 raft_consensus.cc:482] T 211fa6a23bda4c11bb3106080c3ace40 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:15.739041 32108 raft_consensus.cc:504] T 211fa6a23bda4c11bb3106080c3ace40 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:15.739389 32108 leader_election.cc:288] T 211fa6a23bda4c11bb3106080c3ace40 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:15.739679 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "211fa6a23bda4c11bb3106080c3ace40" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:15.739765 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "211fa6a23bda4c11bb3106080c3ace40" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:15.739800 31625 raft_consensus.cc:2454] T 211fa6a23bda4c11bb3106080c3ace40 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:15.739877 29446 raft_consensus.cc:2454] T 211fa6a23bda4c11bb3106080c3ace40 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:15.740243 31429 leader_election.cc:302] T 211fa6a23bda4c11bb3106080c3ace40 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, 315915efcf2b413ca7f9d874338e1b2d; no voters:
I20240502 20:46:15.740406 32108 raft_consensus.cc:2781] T 211fa6a23bda4c11bb3106080c3ace40 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:15.740459 32108 raft_consensus.cc:482] T 211fa6a23bda4c11bb3106080c3ace40 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:15.740500 32108 raft_consensus.cc:3037] T 211fa6a23bda4c11bb3106080c3ace40 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:15.741439 32108 raft_consensus.cc:504] T 211fa6a23bda4c11bb3106080c3ace40 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:15.741799 32108 leader_election.cc:288] T 211fa6a23bda4c11bb3106080c3ace40 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:15.742062 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "211fa6a23bda4c11bb3106080c3ace40" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:15.742173 29446 raft_consensus.cc:3037] T 211fa6a23bda4c11bb3106080c3ace40 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:15.742357 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "211fa6a23bda4c11bb3106080c3ace40" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:15.742484 31625 raft_consensus.cc:3037] T 211fa6a23bda4c11bb3106080c3ace40 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:15.743136 29446 raft_consensus.cc:2454] T 211fa6a23bda4c11bb3106080c3ace40 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:15.743403 31430 leader_election.cc:302] T 211fa6a23bda4c11bb3106080c3ace40 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:15.743463 31625 raft_consensus.cc:2454] T 211fa6a23bda4c11bb3106080c3ace40 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:15.744474 32108 raft_consensus.cc:2781] T 211fa6a23bda4c11bb3106080c3ace40 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:15.744554 32108 raft_consensus.cc:686] T 211fa6a23bda4c11bb3106080c3ace40 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Becoming Leader. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Running, Role: LEADER
I20240502 20:46:15.744684 32108 consensus_queue.cc:237] T 211fa6a23bda4c11bb3106080c3ace40 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:15.746251 29038 catalog_manager.cc:5484] T 211fa6a23bda4c11bb3106080c3ace40 P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69). New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:15.758664 29446 consensus_queue.cc:237] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:15.759908 31625 raft_consensus.cc:1261] T 846f1d6e39e642cfaabf96ad0c52366d P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:15.759920 29574 raft_consensus.cc:1261] T 846f1d6e39e642cfaabf96ad0c52366d P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:15.760267 32571 consensus_queue.cc:1035] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:15.760466 915 consensus_queue.cc:1035] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:15.762732 29574 raft_consensus.cc:2932] T 846f1d6e39e642cfaabf96ad0c52366d P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:15.762779 1183 raft_consensus.cc:2932] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:15.764111 31625 raft_consensus.cc:2932] T 846f1d6e39e642cfaabf96ad0c52366d P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:15.765667 29038 catalog_manager.cc:5484] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index 4 to 5, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) evicted. New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:15.768596 29162 tablet_service.cc:1514] Processing DeleteTablet for tablet 846f1d6e39e642cfaabf96ad0c52366d with delete_type TABLET_DATA_TOMBSTONED (TS 7796ea3fba294d17a34386873969431d not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:15.768862 943 tablet_replica.cc:330] T 846f1d6e39e642cfaabf96ad0c52366d P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:15.769017 943 raft_consensus.cc:2229] T 846f1d6e39e642cfaabf96ad0c52366d P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:15.769172 943 raft_consensus.cc:2258] T 846f1d6e39e642cfaabf96ad0c52366d P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:15.769701 943 ts_tablet_manager.cc:1875] T 846f1d6e39e642cfaabf96ad0c52366d P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:15.771754 943 ts_tablet_manager.cc:1888] T 846f1d6e39e642cfaabf96ad0c52366d P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 20:46:15.771806 943 log.cc:1196] T 846f1d6e39e642cfaabf96ad0c52366d P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/846f1d6e39e642cfaabf96ad0c52366d
I20240502 20:46:15.772197 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 846f1d6e39e642cfaabf96ad0c52366d (table rebalance_test_table_13 [id=08ce6b9eef5e4b2da494180f1ee1852b]) successfully deleted
I20240502 20:46:15.776356 29037 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:15.782434 893 tablet.cc:1722] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:15.782876 1099 tablet.cc:1722] T 994d5557455c4708b108dba74ba5fc17 P db17f26a73f3447b92aa9a947e3a7057: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:15.785009 1092 tablet.cc:1722] T 994d5557455c4708b108dba74ba5fc17 P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:15.793426 29037 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=de8c6ae97d584d2bacde6f7cf5f7d23f] alter complete (version 3)
I20240502 20:46:15.824023 692 raft_consensus.cc:482] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:15.824167 692 raft_consensus.cc:504] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:15.824676 692 leader_election.cc:288] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:15.825089 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f14cf6deefec46c7adb598f6d7cfd649" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:15.825259 31494 raft_consensus.cc:2454] T f14cf6deefec46c7adb598f6d7cfd649 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:15.825551 31692 leader_election.cc:302] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:15.825722 692 raft_consensus.cc:2781] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:15.825794 692 raft_consensus.cc:482] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:15.825842 692 raft_consensus.cc:3037] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:15.826978 692 raft_consensus.cc:504] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:15.827347 692 leader_election.cc:288] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:15.827767 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f14cf6deefec46c7adb598f6d7cfd649" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:15.827886 31494 raft_consensus.cc:3037] T f14cf6deefec46c7adb598f6d7cfd649 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:15.828891 31494 raft_consensus.cc:2454] T f14cf6deefec46c7adb598f6d7cfd649 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:15.829224 31692 leader_election.cc:302] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:15.829406 692 raft_consensus.cc:2781] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:15.829478 692 raft_consensus.cc:686] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Becoming Leader. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Running, Role: LEADER
I20240502 20:46:15.829599 692 consensus_queue.cc:237] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:15.831385 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f14cf6deefec46c7adb598f6d7cfd649" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:15.831530 29574 raft_consensus.cc:2454] T f14cf6deefec46c7adb598f6d7cfd649 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:15.831734 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f14cf6deefec46c7adb598f6d7cfd649" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:15.831833 29577 raft_consensus.cc:3037] T f14cf6deefec46c7adb598f6d7cfd649 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:15.831831 29041 catalog_manager.cc:5484] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: term changed from 0 to 1, leader changed from <none> to e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71). New cstate: current_term: 1 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:15.832831 29577 raft_consensus.cc:2454] T f14cf6deefec46c7adb598f6d7cfd649 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:15.896272 29039 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:15.896430 29039 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:15.898591 29039 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 994d5557455c4708b108dba74ba5fc17
I20240502 20:46:15.899179 31867 tablet_service.cc:1514] Processing DeleteTablet for tablet 994d5557455c4708b108dba74ba5fc17 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:15 UTC) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:15.899176 29557 tablet_service.cc:1514] Processing DeleteTablet for tablet 994d5557455c4708b108dba74ba5fc17 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:15 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:15.899231 29293 tablet_service.cc:1514] Processing DeleteTablet for tablet 994d5557455c4708b108dba74ba5fc17 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:15 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:15.899399 32088 tablet_replica.cc:330] T 994d5557455c4708b108dba74ba5fc17 P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:15.899449 738 tablet_replica.cc:330] T 994d5557455c4708b108dba74ba5fc17 P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:15.899516 1225 tablet_replica.cc:330] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:15.899545 32088 raft_consensus.cc:2229] T 994d5557455c4708b108dba74ba5fc17 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:15.899596 738 raft_consensus.cc:2229] T 994d5557455c4708b108dba74ba5fc17 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:15.899627 1225 raft_consensus.cc:2229] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:15.899679 32088 raft_consensus.cc:2258] T 994d5557455c4708b108dba74ba5fc17 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:15.899734 738 raft_consensus.cc:2258] T 994d5557455c4708b108dba74ba5fc17 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:15.899811 1225 raft_consensus.cc:2258] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:15.900211 32088 ts_tablet_manager.cc:1875] T 994d5557455c4708b108dba74ba5fc17 P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:15.900385 738 ts_tablet_manager.cc:1875] T 994d5557455c4708b108dba74ba5fc17 P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:15.900382 1225 ts_tablet_manager.cc:1875] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:15.902019 32088 ts_tablet_manager.cc:1888] T 994d5557455c4708b108dba74ba5fc17 P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:15.902108 32088 log.cc:1196] T 994d5557455c4708b108dba74ba5fc17 P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/994d5557455c4708b108dba74ba5fc17
I20240502 20:46:15.902338 32088 ts_tablet_manager.cc:1909] T 994d5557455c4708b108dba74ba5fc17 P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:15.902433 738 ts_tablet_manager.cc:1888] T 994d5557455c4708b108dba74ba5fc17 P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:15.902447 1225 ts_tablet_manager.cc:1888] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:15.902523 738 log.cc:1196] T 994d5557455c4708b108dba74ba5fc17 P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/994d5557455c4708b108dba74ba5fc17
I20240502 20:46:15.902524 1225 log.cc:1196] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/994d5557455c4708b108dba74ba5fc17
I20240502 20:46:15.902763 738 ts_tablet_manager.cc:1909] T 994d5557455c4708b108dba74ba5fc17 P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:15.902763 1225 ts_tablet_manager.cc:1909] T 994d5557455c4708b108dba74ba5fc17 P e6b60803d0764d228c863e2da65e3691: Deleting consensus metadata
I20240502 20:46:15.903050 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet 994d5557455c4708b108dba74ba5fc17 (table rebalancer_dynamic_table_new_name [id=de8c6ae97d584d2bacde6f7cf5f7d23f]) successfully deleted
I20240502 20:46:15.903342 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet 994d5557455c4708b108dba74ba5fc17 (table rebalancer_dynamic_table_new_name [id=de8c6ae97d584d2bacde6f7cf5f7d23f]) successfully deleted
I20240502 20:46:15.903390 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet 994d5557455c4708b108dba74ba5fc17 (table rebalancer_dynamic_table_new_name [id=de8c6ae97d584d2bacde6f7cf5f7d23f]) successfully deleted
I20240502 20:46:15.969066 32172 raft_consensus.cc:482] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:15.969193 32172 raft_consensus.cc:504] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:15.969581 32172 leader_election.cc:288] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:15.969969 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11ee3029cd744581ba9c5c213ca8fd2b" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:15.970022 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11ee3029cd744581ba9c5c213ca8fd2b" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:15.970122 31757 raft_consensus.cc:2454] T 11ee3029cd744581ba9c5c213ca8fd2b P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:15.970170 31625 raft_consensus.cc:2454] T 11ee3029cd744581ba9c5c213ca8fd2b P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:15.970449 31822 leader_election.cc:302] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:15.970638 32172 raft_consensus.cc:2781] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:15.970693 32172 raft_consensus.cc:482] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:15.970729 32172 raft_consensus.cc:3037] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:15.971760 32172 raft_consensus.cc:504] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:15.972084 32172 leader_election.cc:288] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:15.972402 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11ee3029cd744581ba9c5c213ca8fd2b" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:15.972469 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11ee3029cd744581ba9c5c213ca8fd2b" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:15.972519 31625 raft_consensus.cc:3037] T 11ee3029cd744581ba9c5c213ca8fd2b P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:15.972579 31757 raft_consensus.cc:3037] T 11ee3029cd744581ba9c5c213ca8fd2b P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:15.973477 31625 raft_consensus.cc:2454] T 11ee3029cd744581ba9c5c213ca8fd2b P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:15.973577 31757 raft_consensus.cc:2454] T 11ee3029cd744581ba9c5c213ca8fd2b P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:15.973634 29039 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalance_test_table_18" } modify_external_catalogs: true
I20240502 20:46:15.973769 29039 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalance_test_table_18" } modify_external_catalogs: true
I20240502 20:46:15.973891 31821 leader_election.cc:302] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e6b60803d0764d228c863e2da65e3691, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:15.974061 32172 raft_consensus.cc:2781] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:15.974159 32172 raft_consensus.cc:686] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Becoming Leader. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Running, Role: LEADER
I20240502 20:46:15.974289 32172 consensus_queue.cc:237] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:15.976446 29038 catalog_manager.cc:5484] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6b60803d0764d228c863e2da65e3691 (127.22.45.72). New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:15.976969 29039 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 4 replicas of tablet 62f3301937ba4471afcc327be05486cf
I20240502 20:46:15.977680 29293 tablet_service.cc:1514] Processing DeleteTablet for tablet 62f3301937ba4471afcc327be05486cf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:15 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:15.977738 29039 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 7e6eb9c58e764ab0be9fb330de4c4ddb
I20240502 20:46:15.977891 31867 tablet_service.cc:1514] Processing DeleteTablet for tablet 62f3301937ba4471afcc327be05486cf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:15 UTC) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:15.977892 29162 tablet_service.cc:1514] Processing DeleteTablet for tablet 62f3301937ba4471afcc327be05486cf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:15 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:15.978067 943 tablet_replica.cc:330] T 62f3301937ba4471afcc327be05486cf P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:15.978224 943 raft_consensus.cc:2229] T 62f3301937ba4471afcc327be05486cf P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:15.978400 943 raft_consensus.cc:2258] T 62f3301937ba4471afcc327be05486cf P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:15.978533 29293 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e6eb9c58e764ab0be9fb330de4c4ddb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:15 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:15.978709 29424 tablet_service.cc:1514] Processing DeleteTablet for tablet 62f3301937ba4471afcc327be05486cf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:15 UTC) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:15.978813 738 tablet_replica.cc:330] T 62f3301937ba4471afcc327be05486cf P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:15.978866 29426 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e6eb9c58e764ab0be9fb330de4c4ddb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:15 UTC) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:15.978904 445 tablet_replica.cc:330] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:15.978950 738 raft_consensus.cc:2229] T 62f3301937ba4471afcc327be05486cf P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:15.978996 943 ts_tablet_manager.cc:1875] T 62f3301937ba4471afcc327be05486cf P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:15.979036 445 raft_consensus.cc:2229] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:15.979099 738 raft_consensus.cc:2258] T 62f3301937ba4471afcc327be05486cf P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:15.979265 445 raft_consensus.cc:2258] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:15.979369 29039 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 5fd8d498911640cfb747d981c3e73f06
I20240502 20:46:15.979671 738 ts_tablet_manager.cc:1875] T 62f3301937ba4471afcc327be05486cf P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:15.979770 445 ts_tablet_manager.cc:1875] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:15.979777 29426 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fd8d498911640cfb747d981c3e73f06 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:15 UTC) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:15.980218 29293 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fd8d498911640cfb747d981c3e73f06 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:15 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:15.981012 1225 tablet_replica.cc:330] T 62f3301937ba4471afcc327be05486cf P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:15.981134 1225 raft_consensus.cc:2229] T 62f3301937ba4471afcc327be05486cf P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:15.981230 1225 raft_consensus.cc:2258] T 62f3301937ba4471afcc327be05486cf P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:15.981631 29557 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e6eb9c58e764ab0be9fb330de4c4ddb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:15 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:15.981781 1225 ts_tablet_manager.cc:1875] T 62f3301937ba4471afcc327be05486cf P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:15.981812 32088 tablet_replica.cc:330] T 7e6eb9c58e764ab0be9fb330de4c4ddb P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:15.981921 32088 raft_consensus.cc:2229] T 7e6eb9c58e764ab0be9fb330de4c4ddb P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:15.981926 445 ts_tablet_manager.cc:1888] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:15.981992 445 log.cc:1196] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/62f3301937ba4471afcc327be05486cf
I20240502 20:46:15.982059 32088 raft_consensus.cc:2258] T 7e6eb9c58e764ab0be9fb330de4c4ddb P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:15.982131 29162 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fd8d498911640cfb747d981c3e73f06 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:15 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:15.982236 445 ts_tablet_manager.cc:1909] T 62f3301937ba4471afcc327be05486cf P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting consensus metadata
I20240502 20:46:15.982303 943 ts_tablet_manager.cc:1888] T 62f3301937ba4471afcc327be05486cf P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:15.982362 943 log.cc:1196] T 62f3301937ba4471afcc327be05486cf P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/62f3301937ba4471afcc327be05486cf
I20240502 20:46:15.982537 32088 ts_tablet_manager.cc:1875] T 7e6eb9c58e764ab0be9fb330de4c4ddb P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:15.982673 943 ts_tablet_manager.cc:1909] T 62f3301937ba4471afcc327be05486cf P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:15.983175 943 tablet_replica.cc:330] T 5fd8d498911640cfb747d981c3e73f06 P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:15.983196 445 tablet_replica.cc:330] T 7e6eb9c58e764ab0be9fb330de4c4ddb P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:15.983295 445 raft_consensus.cc:2229] T 7e6eb9c58e764ab0be9fb330de4c4ddb P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:15.983302 943 raft_consensus.cc:2229] T 5fd8d498911640cfb747d981c3e73f06 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:15.983292 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet 62f3301937ba4471afcc327be05486cf (table rebalance_test_table_18 [id=8dca593309e7400190f93e57aecd5c8e]) successfully deleted
I20240502 20:46:15.983415 943 raft_consensus.cc:2258] T 5fd8d498911640cfb747d981c3e73f06 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:15.983551 445 raft_consensus.cc:2258] T 7e6eb9c58e764ab0be9fb330de4c4ddb P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:15.983690 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 62f3301937ba4471afcc327be05486cf (table rebalance_test_table_18 [id=8dca593309e7400190f93e57aecd5c8e]) successfully deleted
I20240502 20:46:15.984021 445 ts_tablet_manager.cc:1875] T 7e6eb9c58e764ab0be9fb330de4c4ddb P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:15.984607 32088 ts_tablet_manager.cc:1888] T 7e6eb9c58e764ab0be9fb330de4c4ddb P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:15.984666 32088 log.cc:1196] T 7e6eb9c58e764ab0be9fb330de4c4ddb P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/7e6eb9c58e764ab0be9fb330de4c4ddb
I20240502 20:46:15.984908 32088 ts_tablet_manager.cc:1909] T 7e6eb9c58e764ab0be9fb330de4c4ddb P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:15.984951 943 ts_tablet_manager.cc:1875] T 5fd8d498911640cfb747d981c3e73f06 P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:15.985451 1225 ts_tablet_manager.cc:1888] T 62f3301937ba4471afcc327be05486cf P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:15.985561 1225 log.cc:1196] T 62f3301937ba4471afcc327be05486cf P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/62f3301937ba4471afcc327be05486cf
I20240502 20:46:15.985786 445 ts_tablet_manager.cc:1888] T 7e6eb9c58e764ab0be9fb330de4c4ddb P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:15.985855 1225 ts_tablet_manager.cc:1909] T 62f3301937ba4471afcc327be05486cf P e6b60803d0764d228c863e2da65e3691: Deleting consensus metadata
I20240502 20:46:15.985880 445 log.cc:1196] T 7e6eb9c58e764ab0be9fb330de4c4ddb P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/7e6eb9c58e764ab0be9fb330de4c4ddb
I20240502 20:46:15.986076 738 ts_tablet_manager.cc:1888] T 62f3301937ba4471afcc327be05486cf P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:15.986127 445 ts_tablet_manager.cc:1909] T 7e6eb9c58e764ab0be9fb330de4c4ddb P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting consensus metadata
I20240502 20:46:15.986153 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet 7e6eb9c58e764ab0be9fb330de4c4ddb (table rebalance_test_table_18 [id=8dca593309e7400190f93e57aecd5c8e]) successfully deleted
I20240502 20:46:15.986178 738 log.cc:1196] T 62f3301937ba4471afcc327be05486cf P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/62f3301937ba4471afcc327be05486cf
I20240502 20:46:15.986672 738 ts_tablet_manager.cc:1909] T 62f3301937ba4471afcc327be05486cf P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:15.986757 445 tablet_replica.cc:330] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:15.986919 445 raft_consensus.cc:2229] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:15.987007 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet 62f3301937ba4471afcc327be05486cf (table rebalance_test_table_18 [id=8dca593309e7400190f93e57aecd5c8e]) successfully deleted
I20240502 20:46:15.987164 445 raft_consensus.cc:2258] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:15.987301 738 tablet_replica.cc:330] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:15.987362 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet 7e6eb9c58e764ab0be9fb330de4c4ddb (table rebalance_test_table_18 [id=8dca593309e7400190f93e57aecd5c8e]) successfully deleted
I20240502 20:46:15.987494 738 raft_consensus.cc:2229] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:15.987654 445 ts_tablet_manager.cc:1875] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:15.987715 738 raft_consensus.cc:2258] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:15.988006 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet 62f3301937ba4471afcc327be05486cf (table rebalance_test_table_18 [id=8dca593309e7400190f93e57aecd5c8e]) successfully deleted
I20240502 20:46:15.988287 738 ts_tablet_manager.cc:1875] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:15.989121 943 ts_tablet_manager.cc:1888] T 5fd8d498911640cfb747d981c3e73f06 P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:15.989245 943 log.cc:1196] T 5fd8d498911640cfb747d981c3e73f06 P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/5fd8d498911640cfb747d981c3e73f06
I20240502 20:46:15.989420 445 ts_tablet_manager.cc:1888] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:15.989491 445 log.cc:1196] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/5fd8d498911640cfb747d981c3e73f06
I20240502 20:46:15.989506 943 ts_tablet_manager.cc:1909] T 5fd8d498911640cfb747d981c3e73f06 P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:15.989758 445 ts_tablet_manager.cc:1909] T 5fd8d498911640cfb747d981c3e73f06 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting consensus metadata
I20240502 20:46:15.990191 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 5fd8d498911640cfb747d981c3e73f06 (table rebalance_test_table_18 [id=8dca593309e7400190f93e57aecd5c8e]) successfully deleted
I20240502 20:46:15.990211 738 ts_tablet_manager.cc:1888] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:15.990278 738 log.cc:1196] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/7e6eb9c58e764ab0be9fb330de4c4ddb
I20240502 20:46:15.990414 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet 5fd8d498911640cfb747d981c3e73f06 (table rebalance_test_table_18 [id=8dca593309e7400190f93e57aecd5c8e]) successfully deleted
I20240502 20:46:15.990582 738 ts_tablet_manager.cc:1909] T 7e6eb9c58e764ab0be9fb330de4c4ddb P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:15.991111 738 tablet_replica.cc:330] T 5fd8d498911640cfb747d981c3e73f06 P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:15.991248 738 raft_consensus.cc:2229] T 5fd8d498911640cfb747d981c3e73f06 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:15.991261 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet 7e6eb9c58e764ab0be9fb330de4c4ddb (table rebalance_test_table_18 [id=8dca593309e7400190f93e57aecd5c8e]) successfully deleted
I20240502 20:46:15.991441 738 raft_consensus.cc:2258] T 5fd8d498911640cfb747d981c3e73f06 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:15.991873 738 ts_tablet_manager.cc:1875] T 5fd8d498911640cfb747d981c3e73f06 P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:15.993736 738 ts_tablet_manager.cc:1888] T 5fd8d498911640cfb747d981c3e73f06 P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 20:46:15.993857 738 log.cc:1196] T 5fd8d498911640cfb747d981c3e73f06 P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/5fd8d498911640cfb747d981c3e73f06
I20240502 20:46:15.994437 738 ts_tablet_manager.cc:1909] T 5fd8d498911640cfb747d981c3e73f06 P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:15.995075 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet 5fd8d498911640cfb747d981c3e73f06 (table rebalance_test_table_18 [id=8dca593309e7400190f93e57aecd5c8e]) successfully deleted
I20240502 20:46:16.000881 29039 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:16.007449 31602 tablet_service.cc:1467] Processing CreateTablet for tablet 75d910af410a43ca8b9ff4c7ea737773 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=3ef2607118b441b4b6682450f6395f6c]), partition=
I20240502 20:46:16.007622 31475 tablet_service.cc:1467] Processing CreateTablet for tablet 75d910af410a43ca8b9ff4c7ea737773 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=3ef2607118b441b4b6682450f6395f6c]), partition=
I20240502 20:46:16.007759 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75d910af410a43ca8b9ff4c7ea737773. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:16.008291 31474 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:16.010178 1151 tablet_bootstrap.cc:492] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:16.011173 1151 tablet_bootstrap.cc:654] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:16.013834 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 75d910af410a43ca8b9ff4c7ea737773 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=3ef2607118b441b4b6682450f6395f6c]), partition=
I20240502 20:46:16.014084 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75d910af410a43ca8b9ff4c7ea737773. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:16.015780 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75d910af410a43ca8b9ff4c7ea737773. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:16.017922 1154 tablet_bootstrap.cc:492] T 75d910af410a43ca8b9ff4c7ea737773 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:16.018764 1154 tablet_bootstrap.cc:654] T 75d910af410a43ca8b9ff4c7ea737773 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:16.020699 29293 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:16.023135 681 tablet_bootstrap.cc:492] T 75d910af410a43ca8b9ff4c7ea737773 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:16.024015 681 tablet_bootstrap.cc:654] T 75d910af410a43ca8b9ff4c7ea737773 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:16.026623 1154 tablet_bootstrap.cc:492] T 75d910af410a43ca8b9ff4c7ea737773 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:16.026723 1154 ts_tablet_manager.cc:1367] T 75d910af410a43ca8b9ff4c7ea737773 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 20:46:16.027216 1154 raft_consensus.cc:348] T 75d910af410a43ca8b9ff4c7ea737773 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.027326 1154 raft_consensus.cc:374] T 75d910af410a43ca8b9ff4c7ea737773 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:16.027356 1154 raft_consensus.cc:729] T 75d910af410a43ca8b9ff4c7ea737773 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:16.027460 1154 consensus_queue.cc:260] T 75d910af410a43ca8b9ff4c7ea737773 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.027735 1154 ts_tablet_manager.cc:1398] T 75d910af410a43ca8b9ff4c7ea737773 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:16.028337 29162 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:16.028785 29557 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:16.033891 29426 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:16.043815 31737 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:46:16.047524 1151 tablet_bootstrap.cc:492] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:16.047629 1151 ts_tablet_manager.cc:1367] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.037s user 0.002s sys 0.000s
I20240502 20:46:16.047683 681 tablet_bootstrap.cc:492] T 75d910af410a43ca8b9ff4c7ea737773 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:16.047786 681 ts_tablet_manager.cc:1367] T 75d910af410a43ca8b9ff4c7ea737773 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.025s user 0.001s sys 0.001s
I20240502 20:46:16.048099 1151 raft_consensus.cc:348] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.048209 1151 raft_consensus.cc:374] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:16.048238 1151 raft_consensus.cc:729] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:16.048221 681 raft_consensus.cc:348] T 75d910af410a43ca8b9ff4c7ea737773 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.048322 681 raft_consensus.cc:374] T 75d910af410a43ca8b9ff4c7ea737773 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:16.048348 681 raft_consensus.cc:729] T 75d910af410a43ca8b9ff4c7ea737773 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:16.048344 1151 consensus_queue.cc:260] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.048458 681 consensus_queue.cc:260] T 75d910af410a43ca8b9ff4c7ea737773 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.048585 1151 ts_tablet_manager.cc:1398] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:16.048708 681 ts_tablet_manager.cc:1398] T 75d910af410a43ca8b9ff4c7ea737773 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 20:46:16.052953 31602 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 20:46:16.071029 31867 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:16.197932 977 consensus_queue.cc:1035] T 211fa6a23bda4c11bb3106080c3ace40 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:16.201208 32108 consensus_queue.cc:1035] T 211fa6a23bda4c11bb3106080c3ace40 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:16.217341 1055 raft_consensus.cc:482] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:16.217464 1055 raft_consensus.cc:504] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.217842 1055 leader_election.cc:288] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:16.218230 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75d910af410a43ca8b9ff4c7ea737773" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:16.218214 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75d910af410a43ca8b9ff4c7ea737773" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:16.218364 31888 raft_consensus.cc:2454] T 75d910af410a43ca8b9ff4c7ea737773 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:16.218364 31494 raft_consensus.cc:2454] T 75d910af410a43ca8b9ff4c7ea737773 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:16.218799 31561 leader_election.cc:302] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, 315915efcf2b413ca7f9d874338e1b2d; no voters:
I20240502 20:46:16.218984 1055 raft_consensus.cc:2781] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:16.219063 1055 raft_consensus.cc:482] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:16.219110 1055 raft_consensus.cc:3037] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:16.220009 1055 raft_consensus.cc:504] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.220314 1055 leader_election.cc:288] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Requested vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:16.220655 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75d910af410a43ca8b9ff4c7ea737773" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:16.220767 31888 raft_consensus.cc:3037] T 75d910af410a43ca8b9ff4c7ea737773 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:16.220741 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75d910af410a43ca8b9ff4c7ea737773" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:16.220858 31494 raft_consensus.cc:3037] T 75d910af410a43ca8b9ff4c7ea737773 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:16.221710 31888 raft_consensus.cc:2454] T 75d910af410a43ca8b9ff4c7ea737773 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:16.221843 31494 raft_consensus.cc:2454] T 75d910af410a43ca8b9ff4c7ea737773 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:16.221988 31560 leader_election.cc:302] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:16.222162 1055 raft_consensus.cc:2781] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:16.222301 1055 raft_consensus.cc:686] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Becoming Leader. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Running, Role: LEADER
I20240502 20:46:16.222429 1055 consensus_queue.cc:237] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.224083 29041 catalog_manager.cc:5484] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70). New cstate: current_term: 1 leader_uuid: "315915efcf2b413ca7f9d874338e1b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:16.242946 29293 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:16.247205 31602 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:16.247249 29557 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:16.249467 29162 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:16.254097 31867 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:16.261046 31475 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:16.261382 29426 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:16.265864 692 consensus_queue.cc:1035] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:16.278160 31737 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:46:16.292774 692 consensus_queue.cc:1035] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:16.326782 29446 consensus_queue.cc:237] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:16.328040 29577 raft_consensus.cc:1261] T 2d72ef6e7e8d473baaff8f527c02332b P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:16.328105 31625 raft_consensus.cc:1261] T 2d72ef6e7e8d473baaff8f527c02332b P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:16.328532 915 consensus_queue.cc:1035] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:16.328768 1298 consensus_queue.cc:1035] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:16.330739 1320 raft_consensus.cc:2932] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:16.331259 31626 raft_consensus.cc:2932] T 2d72ef6e7e8d473baaff8f527c02332b P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:16.331259 29577 raft_consensus.cc:2932] T 2d72ef6e7e8d473baaff8f527c02332b P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:16.334090 29041 catalog_manager.cc:5484] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index 4 to 5, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) evicted. New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:16.337136 29162 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d72ef6e7e8d473baaff8f527c02332b with delete_type TABLET_DATA_TOMBSTONED (TS 7796ea3fba294d17a34386873969431d not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:16.337389 943 tablet_replica.cc:330] T 2d72ef6e7e8d473baaff8f527c02332b P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:16.337548 943 raft_consensus.cc:2229] T 2d72ef6e7e8d473baaff8f527c02332b P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:16.337719 943 raft_consensus.cc:2258] T 2d72ef6e7e8d473baaff8f527c02332b P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:16.338230 943 ts_tablet_manager.cc:1875] T 2d72ef6e7e8d473baaff8f527c02332b P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:16.340654 943 ts_tablet_manager.cc:1888] T 2d72ef6e7e8d473baaff8f527c02332b P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 20:46:16.340734 943 log.cc:1196] T 2d72ef6e7e8d473baaff8f527c02332b P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/2d72ef6e7e8d473baaff8f527c02332b
I20240502 20:46:16.341292 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 2d72ef6e7e8d473baaff8f527c02332b (table rebalance_test_table_3 [id=aec26ee01a7a4b05887071d6b1783fb0]) successfully deleted
I20240502 20:46:16.372467 29038 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:16.376852 31494 raft_consensus.cc:1261] T 75d910af410a43ca8b9ff4c7ea737773 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Refusing update from remote peer 315915efcf2b413ca7f9d874338e1b2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:16.376906 31888 raft_consensus.cc:1261] T 75d910af410a43ca8b9ff4c7ea737773 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer 315915efcf2b413ca7f9d874338e1b2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:16.377334 1284 consensus_queue.cc:1035] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:16.377529 1055 consensus_queue.cc:1035] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:16.379808 29038 catalog_manager.cc:5576] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887) does not have the latest schema for tablet 75d910af410a43ca8b9ff4c7ea737773 (table rebalancer_dynamic_table [id=3ef2607118b441b4b6682450f6395f6c]). Expected version 1 got 0
I20240502 20:46:16.381466 29038 catalog_manager.cc:5576] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207) does not have the latest schema for tablet 75d910af410a43ca8b9ff4c7ea737773 (table rebalancer_dynamic_table [id=3ef2607118b441b4b6682450f6395f6c]). Expected version 1 got 0
I20240502 20:46:16.382797 1340 tablet.cc:1722] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:16.382858 29038 catalog_manager.cc:5576] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887) does not have the latest schema for tablet 75d910af410a43ca8b9ff4c7ea737773 (table rebalancer_dynamic_table [id=3ef2607118b441b4b6682450f6395f6c]). Expected version 1 got 0
I20240502 20:46:16.383227 1340 tablet.cc:1620] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:16.383358 1342 tablet.cc:1722] T 75d910af410a43ca8b9ff4c7ea737773 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:16.383723 1342 tablet.cc:1620] T 75d910af410a43ca8b9ff4c7ea737773 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:16.384171 1341 tablet.cc:1722] T 75d910af410a43ca8b9ff4c7ea737773 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:16.384585 1341 tablet.cc:1620] T 75d910af410a43ca8b9ff4c7ea737773 P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:16.384589 29037 catalog_manager.cc:5576] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207) does not have the latest schema for tablet 75d910af410a43ca8b9ff4c7ea737773 (table rebalancer_dynamic_table [id=3ef2607118b441b4b6682450f6395f6c]). Expected version 1 got 0
I20240502 20:46:16.388082 29037 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=3ef2607118b441b4b6682450f6395f6c] alter complete (version 1)
I20240502 20:46:16.401751 32172 consensus_queue.cc:1035] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:16.405833 822 consensus_queue.cc:1035] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:16.436278 29293 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:16.437033 31602 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:16.439393 29557 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:16.443452 29162 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:16.444754 31475 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:16.453838 29426 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:16.458667 31737 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:46:16.484050 31867 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:16.489882 29037 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_12"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:16.491403 29039 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:16.497787 31737 tablet_service.cc:1467] Processing CreateTablet for tablet 6436a7d43dab46bca948438f832245ed (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=640d5c5611d54c78844cd067d1b6828a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:16.498072 31736 tablet_service.cc:1467] Processing CreateTablet for tablet 358e4c761223404fbf0f467f38ce9b25 (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=640d5c5611d54c78844cd067d1b6828a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:16.498132 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6436a7d43dab46bca948438f832245ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:16.498996 31735 tablet_service.cc:1467] Processing CreateTablet for tablet 82f57a76ce934ebab6c68948c543e0cb (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=640d5c5611d54c78844cd067d1b6828a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:16.499250 31735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82f57a76ce934ebab6c68948c543e0cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:16.499218 1340 tablet.cc:1722] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:16.499516 1340 tablet.cc:1620] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:16.500397 1341 tablet.cc:1722] T 75d910af410a43ca8b9ff4c7ea737773 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:16.500681 1341 tablet.cc:1620] T 75d910af410a43ca8b9ff4c7ea737773 P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:16.500828 31736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 358e4c761223404fbf0f467f38ce9b25. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:16.501314 31475 tablet_service.cc:1467] Processing CreateTablet for tablet 358e4c761223404fbf0f467f38ce9b25 (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=640d5c5611d54c78844cd067d1b6828a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:16.501540 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 358e4c761223404fbf0f467f38ce9b25. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:16.503425 1154 tablet_bootstrap.cc:492] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:16.503630 31474 tablet_service.cc:1467] Processing CreateTablet for tablet 82f57a76ce934ebab6c68948c543e0cb (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=640d5c5611d54c78844cd067d1b6828a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:16.503830 31474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82f57a76ce934ebab6c68948c543e0cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:16.504343 1154 tablet_bootstrap.cc:654] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:16.504590 29557 tablet_service.cc:1467] Processing CreateTablet for tablet 82f57a76ce934ebab6c68948c543e0cb (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=640d5c5611d54c78844cd067d1b6828a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:16.504868 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82f57a76ce934ebab6c68948c543e0cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:16.505008 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 6436a7d43dab46bca948438f832245ed (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=640d5c5611d54c78844cd067d1b6828a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:16.505254 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6436a7d43dab46bca948438f832245ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:16.507361 681 tablet_bootstrap.cc:492] T 6436a7d43dab46bca948438f832245ed P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:16.508203 681 tablet_bootstrap.cc:654] T 6436a7d43dab46bca948438f832245ed P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:16.509275 29038 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=3ef2607118b441b4b6682450f6395f6c] alter complete (version 2)
I20240502 20:46:16.510623 1380 tablet_bootstrap.cc:492] T 6436a7d43dab46bca948438f832245ed P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:16.510839 1154 tablet_bootstrap.cc:492] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:16.510932 1154 ts_tablet_manager.cc:1367] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 20:46:16.511410 31606 tablet_service.cc:1467] Processing CreateTablet for tablet 358e4c761223404fbf0f467f38ce9b25 (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=640d5c5611d54c78844cd067d1b6828a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:16.511660 1380 tablet_bootstrap.cc:654] T 6436a7d43dab46bca948438f832245ed P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:16.511725 31606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 358e4c761223404fbf0f467f38ce9b25. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:16.512055 1342 tablet.cc:1722] T 75d910af410a43ca8b9ff4c7ea737773 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:16.512413 1342 tablet.cc:1620] T 75d910af410a43ca8b9ff4c7ea737773 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:16.512818 1154 raft_consensus.cc:348] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.512949 1154 raft_consensus.cc:374] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:16.513008 1154 raft_consensus.cc:729] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:16.513142 1154 consensus_queue.cc:260] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.513805 1154 ts_tablet_manager.cc:1398] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:16.513975 1154 tablet_bootstrap.cc:492] T 82f57a76ce934ebab6c68948c543e0cb P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:16.513979 1151 tablet_bootstrap.cc:492] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:16.514923 1151 tablet_bootstrap.cc:654] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:16.515163 1154 tablet_bootstrap.cc:654] T 82f57a76ce934ebab6c68948c543e0cb P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:16.516076 1151 tablet_bootstrap.cc:492] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:16.516160 1151 ts_tablet_manager.cc:1367] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:16.516569 31602 tablet_service.cc:1467] Processing CreateTablet for tablet 6436a7d43dab46bca948438f832245ed (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=640d5c5611d54c78844cd067d1b6828a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:16.516281 1154 tablet_bootstrap.cc:492] T 82f57a76ce934ebab6c68948c543e0cb P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:16.516767 1154 ts_tablet_manager.cc:1367] T 82f57a76ce934ebab6c68948c543e0cb P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:16.516824 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6436a7d43dab46bca948438f832245ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:16.517196 1154 raft_consensus.cc:348] T 82f57a76ce934ebab6c68948c543e0cb P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.517300 1154 raft_consensus.cc:374] T 82f57a76ce934ebab6c68948c543e0cb P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:16.517326 1154 raft_consensus.cc:729] T 82f57a76ce934ebab6c68948c543e0cb P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:16.517463 1154 consensus_queue.cc:260] T 82f57a76ce934ebab6c68948c543e0cb P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.517736 1154 ts_tablet_manager.cc:1398] T 82f57a76ce934ebab6c68948c543e0cb P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:16.518065 1382 tablet_bootstrap.cc:492] T 82f57a76ce934ebab6c68948c543e0cb P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:16.518983 1382 tablet_bootstrap.cc:654] T 82f57a76ce934ebab6c68948c543e0cb P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:16.520256 1382 tablet_bootstrap.cc:492] T 82f57a76ce934ebab6c68948c543e0cb P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:16.520349 1382 ts_tablet_manager.cc:1367] T 82f57a76ce934ebab6c68948c543e0cb P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:16.516579 1151 raft_consensus.cc:348] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.520490 1151 raft_consensus.cc:374] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:16.520521 1151 raft_consensus.cc:729] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:16.520649 1151 consensus_queue.cc:260] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.520762 1382 raft_consensus.cc:348] T 82f57a76ce934ebab6c68948c543e0cb P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.520900 1382 raft_consensus.cc:374] T 82f57a76ce934ebab6c68948c543e0cb P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:16.520911 1151 ts_tablet_manager.cc:1398] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 20:46:16.520929 1382 raft_consensus.cc:729] T 82f57a76ce934ebab6c68948c543e0cb P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:16.521057 1151 tablet_bootstrap.cc:492] T 6436a7d43dab46bca948438f832245ed P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:16.521042 1382 consensus_queue.cc:260] T 82f57a76ce934ebab6c68948c543e0cb P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.521328 1382 ts_tablet_manager.cc:1398] T 82f57a76ce934ebab6c68948c543e0cb P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:16.521903 1151 tablet_bootstrap.cc:654] T 6436a7d43dab46bca948438f832245ed P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:16.524555 681 tablet_bootstrap.cc:492] T 6436a7d43dab46bca948438f832245ed P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:16.524653 681 ts_tablet_manager.cc:1367] T 6436a7d43dab46bca948438f832245ed P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.001s
I20240502 20:46:16.525076 681 raft_consensus.cc:348] T 6436a7d43dab46bca948438f832245ed P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:16.525177 681 raft_consensus.cc:374] T 6436a7d43dab46bca948438f832245ed P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:16.525202 681 raft_consensus.cc:729] T 6436a7d43dab46bca948438f832245ed P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:16.525326 681 consensus_queue.cc:260] T 6436a7d43dab46bca948438f832245ed P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:16.525575 681 ts_tablet_manager.cc:1398] T 6436a7d43dab46bca948438f832245ed P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:16.526356 1380 tablet_bootstrap.cc:492] T 6436a7d43dab46bca948438f832245ed P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:16.526485 1380 ts_tablet_manager.cc:1367] T 6436a7d43dab46bca948438f832245ed P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240502 20:46:16.526935 1380 raft_consensus.cc:348] T 6436a7d43dab46bca948438f832245ed P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:16.527031 1380 raft_consensus.cc:374] T 6436a7d43dab46bca948438f832245ed P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:16.527057 1380 raft_consensus.cc:729] T 6436a7d43dab46bca948438f832245ed P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:16.527180 1380 consensus_queue.cc:260] T 6436a7d43dab46bca948438f832245ed P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:16.527442 1380 ts_tablet_manager.cc:1398] T 6436a7d43dab46bca948438f832245ed P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:16.527619 1380 tablet_bootstrap.cc:492] T 82f57a76ce934ebab6c68948c543e0cb P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:16.528470 1380 tablet_bootstrap.cc:654] T 82f57a76ce934ebab6c68948c543e0cb P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:16.536310 1380 tablet_bootstrap.cc:492] T 82f57a76ce934ebab6c68948c543e0cb P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:16.536440 1380 ts_tablet_manager.cc:1367] T 82f57a76ce934ebab6c68948c543e0cb P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 20:46:16.536934 1380 raft_consensus.cc:348] T 82f57a76ce934ebab6c68948c543e0cb P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.537050 1380 raft_consensus.cc:374] T 82f57a76ce934ebab6c68948c543e0cb P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:16.537082 1380 raft_consensus.cc:729] T 82f57a76ce934ebab6c68948c543e0cb P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:16.537204 1380 consensus_queue.cc:260] T 82f57a76ce934ebab6c68948c543e0cb P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.537482 1380 ts_tablet_manager.cc:1398] T 82f57a76ce934ebab6c68948c543e0cb P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:16.537650 1380 tablet_bootstrap.cc:492] T 358e4c761223404fbf0f467f38ce9b25 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:16.540887 1380 tablet_bootstrap.cc:654] T 358e4c761223404fbf0f467f38ce9b25 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:16.541976 1151 tablet_bootstrap.cc:492] T 6436a7d43dab46bca948438f832245ed P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:16.542068 1151 ts_tablet_manager.cc:1367] T 6436a7d43dab46bca948438f832245ed P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20240502 20:46:16.542114 1380 tablet_bootstrap.cc:492] T 358e4c761223404fbf0f467f38ce9b25 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:16.542179 1380 ts_tablet_manager.cc:1367] T 358e4c761223404fbf0f467f38ce9b25 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 20:46:16.542500 1151 raft_consensus.cc:348] T 6436a7d43dab46bca948438f832245ed P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:16.542609 1151 raft_consensus.cc:374] T 6436a7d43dab46bca948438f832245ed P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:16.542580 1380 raft_consensus.cc:348] T 358e4c761223404fbf0f467f38ce9b25 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.542634 1151 raft_consensus.cc:729] T 6436a7d43dab46bca948438f832245ed P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:16.542660 1380 raft_consensus.cc:374] T 358e4c761223404fbf0f467f38ce9b25 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:16.542687 1380 raft_consensus.cc:729] T 358e4c761223404fbf0f467f38ce9b25 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:16.542750 1151 consensus_queue.cc:260] T 6436a7d43dab46bca948438f832245ed P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:16.542843 1380 consensus_queue.cc:260] T 358e4c761223404fbf0f467f38ce9b25 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.542989 1151 ts_tablet_manager.cc:1398] T 6436a7d43dab46bca948438f832245ed P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:16.543862 1380 ts_tablet_manager.cc:1398] T 358e4c761223404fbf0f467f38ce9b25 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:16.555130 29175 consensus_queue.cc:237] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:16.556733 29577 raft_consensus.cc:1261] T 616e6b2cacc54145a29a10cc8584bbdf P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:16.556852 31888 raft_consensus.cc:1261] T 616e6b2cacc54145a29a10cc8584bbdf P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:16.557242 32213 consensus_queue.cc:1035] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:16.557442 1093 consensus_queue.cc:1035] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:16.559330 1093 raft_consensus.cc:2932] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:16.560828 29577 raft_consensus.cc:2932] T 616e6b2cacc54145a29a10cc8584bbdf P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:16.561648 31888 raft_consensus.cc:2932] T 616e6b2cacc54145a29a10cc8584bbdf P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:16.561973 29039 catalog_manager.cc:5484] T 616e6b2cacc54145a29a10cc8584bbdf P 7796ea3fba294d17a34386873969431d reported cstate change: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:16.565411 29426 tablet_service.cc:1514] Processing DeleteTablet for tablet 616e6b2cacc54145a29a10cc8584bbdf with delete_type TABLET_DATA_TOMBSTONED (TS dbcdc6ac635b44b185c3eba63f66e9d6 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:16.565769 1412 tablet_replica.cc:330] T 616e6b2cacc54145a29a10cc8584bbdf P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:16.565971 1412 raft_consensus.cc:2229] T 616e6b2cacc54145a29a10cc8584bbdf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:16.566125 1412 raft_consensus.cc:2258] T 616e6b2cacc54145a29a10cc8584bbdf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:16.566679 1412 ts_tablet_manager.cc:1875] T 616e6b2cacc54145a29a10cc8584bbdf P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:16.569962 1412 ts_tablet_manager.cc:1888] T 616e6b2cacc54145a29a10cc8584bbdf P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 20:46:16.570031 1412 log.cc:1196] T 616e6b2cacc54145a29a10cc8584bbdf P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/616e6b2cacc54145a29a10cc8584bbdf
I20240502 20:46:16.570483 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet 616e6b2cacc54145a29a10cc8584bbdf (table rebalance_test_table_13 [id=08ce6b9eef5e4b2da494180f1ee1852b]) successfully deleted
I20240502 20:46:16.621253 29037 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:16.630743 1340 tablet.cc:1722] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:16.631623 1342 tablet.cc:1722] T 75d910af410a43ca8b9ff4c7ea737773 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:16.633882 1341 tablet.cc:1722] T 75d910af410a43ca8b9ff4c7ea737773 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:16.635943 29037 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=3ef2607118b441b4b6682450f6395f6c] alter complete (version 3)
I20240502 20:46:16.651329 1379 raft_consensus.cc:482] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:16.651470 1379 raft_consensus.cc:504] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.651971 1379 leader_election.cc:288] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:16.652547 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "358e4c761223404fbf0f467f38ce9b25" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:16.652627 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "358e4c761223404fbf0f467f38ce9b25" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:16.652707 31757 raft_consensus.cc:2454] T 358e4c761223404fbf0f467f38ce9b25 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:16.652797 31494 raft_consensus.cc:2454] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:16.653067 31559 leader_election.cc:302] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:16.653606 1379 raft_consensus.cc:2781] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:16.653667 1379 raft_consensus.cc:482] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:16.653710 1379 raft_consensus.cc:3037] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:16.654774 1379 raft_consensus.cc:504] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.655215 1379 leader_election.cc:288] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Requested vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:16.655643 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "358e4c761223404fbf0f467f38ce9b25" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:16.655771 31494 raft_consensus.cc:3037] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:16.655802 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "358e4c761223404fbf0f467f38ce9b25" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:16.655910 31757 raft_consensus.cc:3037] T 358e4c761223404fbf0f467f38ce9b25 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:16.656780 31494 raft_consensus.cc:2454] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:16.656805 31757 raft_consensus.cc:2454] T 358e4c761223404fbf0f467f38ce9b25 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:16.657073 31561 leader_election.cc:302] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, 315915efcf2b413ca7f9d874338e1b2d; no voters:
I20240502 20:46:16.657258 1379 raft_consensus.cc:2781] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:16.657347 1379 raft_consensus.cc:686] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Becoming Leader. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Running, Role: LEADER
I20240502 20:46:16.657485 1379 consensus_queue.cc:237] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.659549 29036 catalog_manager.cc:5484] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70). New cstate: current_term: 1 leader_uuid: "315915efcf2b413ca7f9d874338e1b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:16.696497 692 raft_consensus.cc:482] T 82f57a76ce934ebab6c68948c543e0cb P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:16.696630 692 raft_consensus.cc:504] T 82f57a76ce934ebab6c68948c543e0cb P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.697124 692 leader_election.cc:288] T 82f57a76ce934ebab6c68948c543e0cb P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:16.697477 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82f57a76ce934ebab6c68948c543e0cb" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:16.697599 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82f57a76ce934ebab6c68948c543e0cb" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:16.697644 29577 raft_consensus.cc:2454] T 82f57a76ce934ebab6c68948c543e0cb P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:16.697726 31494 raft_consensus.cc:2454] T 82f57a76ce934ebab6c68948c543e0cb P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:16.697978 31691 leader_election.cc:302] T 82f57a76ce934ebab6c68948c543e0cb P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:16.698160 692 raft_consensus.cc:2781] T 82f57a76ce934ebab6c68948c543e0cb P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:16.698228 692 raft_consensus.cc:482] T 82f57a76ce934ebab6c68948c543e0cb P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:16.698282 692 raft_consensus.cc:3037] T 82f57a76ce934ebab6c68948c543e0cb P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:16.699191 692 raft_consensus.cc:504] T 82f57a76ce934ebab6c68948c543e0cb P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.699537 692 leader_election.cc:288] T 82f57a76ce934ebab6c68948c543e0cb P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:16.699920 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82f57a76ce934ebab6c68948c543e0cb" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:16.699932 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82f57a76ce934ebab6c68948c543e0cb" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:16.700035 31494 raft_consensus.cc:3037] T 82f57a76ce934ebab6c68948c543e0cb P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:16.700058 29577 raft_consensus.cc:3037] T 82f57a76ce934ebab6c68948c543e0cb P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:16.700984 31494 raft_consensus.cc:2454] T 82f57a76ce934ebab6c68948c543e0cb P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:16.701128 29577 raft_consensus.cc:2454] T 82f57a76ce934ebab6c68948c543e0cb P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:16.701382 31692 leader_election.cc:302] T 82f57a76ce934ebab6c68948c543e0cb P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:16.701642 692 raft_consensus.cc:2781] T 82f57a76ce934ebab6c68948c543e0cb P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:16.701745 692 raft_consensus.cc:686] T 82f57a76ce934ebab6c68948c543e0cb P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Becoming Leader. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Running, Role: LEADER
I20240502 20:46:16.701892 692 consensus_queue.cc:237] T 82f57a76ce934ebab6c68948c543e0cb P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.704097 29036 catalog_manager.cc:5484] T 82f57a76ce934ebab6c68948c543e0cb P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: term changed from 0 to 1, leader changed from <none> to e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71). New cstate: current_term: 1 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:16.739210 29036 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:16.739357 29036 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:16.741397 29036 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 75d910af410a43ca8b9ff4c7ea737773
I20240502 20:46:16.742017 31602 tablet_service.cc:1514] Processing DeleteTablet for tablet 75d910af410a43ca8b9ff4c7ea737773 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:16 UTC) from {username='slave'} at 127.0.0.1:58346
I20240502 20:46:16.742240 31867 tablet_service.cc:1514] Processing DeleteTablet for tablet 75d910af410a43ca8b9ff4c7ea737773 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:16 UTC) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:16.742300 31474 tablet_service.cc:1514] Processing DeleteTablet for tablet 75d910af410a43ca8b9ff4c7ea737773 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:16 UTC) from {username='slave'} at 127.0.0.1:40276
I20240502 20:46:16.742548 1449 tablet_replica.cc:330] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d: stopping tablet replica
I20240502 20:46:16.742563 1450 tablet_replica.cc:330] T 75d910af410a43ca8b9ff4c7ea737773 P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:16.742672 1451 tablet_replica.cc:330] T 75d910af410a43ca8b9ff4c7ea737773 P 2a9c1af8cf814030bde56b67ca5c2fbe: stopping tablet replica
I20240502 20:46:16.742709 1450 raft_consensus.cc:2229] T 75d910af410a43ca8b9ff4c7ea737773 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:16.742708 1449 raft_consensus.cc:2229] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:16.742811 1451 raft_consensus.cc:2229] T 75d910af410a43ca8b9ff4c7ea737773 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:16.742835 1450 raft_consensus.cc:2258] T 75d910af410a43ca8b9ff4c7ea737773 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:16.742923 1449 raft_consensus.cc:2258] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:16.742937 1451 raft_consensus.cc:2258] T 75d910af410a43ca8b9ff4c7ea737773 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:16.743458 1451 ts_tablet_manager.cc:1875] T 75d910af410a43ca8b9ff4c7ea737773 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:16.743458 1450 ts_tablet_manager.cc:1875] T 75d910af410a43ca8b9ff4c7ea737773 P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:16.743608 1449 ts_tablet_manager.cc:1875] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:16.745553 1449 ts_tablet_manager.cc:1888] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:16.745622 1449 log.cc:1196] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal/wals/75d910af410a43ca8b9ff4c7ea737773
I20240502 20:46:16.745848 1449 ts_tablet_manager.cc:1909] T 75d910af410a43ca8b9ff4c7ea737773 P 315915efcf2b413ca7f9d874338e1b2d: Deleting consensus metadata
I20240502 20:46:16.745864 1451 ts_tablet_manager.cc:1888] T 75d910af410a43ca8b9ff4c7ea737773 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:16.745930 1451 log.cc:1196] T 75d910af410a43ca8b9ff4c7ea737773 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/wals/75d910af410a43ca8b9ff4c7ea737773
I20240502 20:46:16.745925 1450 ts_tablet_manager.cc:1888] T 75d910af410a43ca8b9ff4c7ea737773 P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:16.745981 1450 log.cc:1196] T 75d910af410a43ca8b9ff4c7ea737773 P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/75d910af410a43ca8b9ff4c7ea737773
I20240502 20:46:16.746259 1451 ts_tablet_manager.cc:1909] T 75d910af410a43ca8b9ff4c7ea737773 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting consensus metadata
I20240502 20:46:16.746259 1450 ts_tablet_manager.cc:1909] T 75d910af410a43ca8b9ff4c7ea737773 P e6b60803d0764d228c863e2da65e3691: Deleting consensus metadata
I20240502 20:46:16.746534 29027 catalog_manager.cc:4830] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): tablet 75d910af410a43ca8b9ff4c7ea737773 (table rebalancer_dynamic_table_new_name [id=3ef2607118b441b4b6682450f6395f6c]) successfully deleted
I20240502 20:46:16.746862 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet 75d910af410a43ca8b9ff4c7ea737773 (table rebalancer_dynamic_table_new_name [id=3ef2607118b441b4b6682450f6395f6c]) successfully deleted
I20240502 20:46:16.746928 29028 catalog_manager.cc:4830] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): tablet 75d910af410a43ca8b9ff4c7ea737773 (table rebalancer_dynamic_table_new_name [id=3ef2607118b441b4b6682450f6395f6c]) successfully deleted
I20240502 20:46:16.782387 692 raft_consensus.cc:482] T 6436a7d43dab46bca948438f832245ed P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:16.782538 692 raft_consensus.cc:504] T 6436a7d43dab46bca948438f832245ed P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:16.782943 692 leader_election.cc:288] T 6436a7d43dab46bca948438f832245ed P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:16.783345 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6436a7d43dab46bca948438f832245ed" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:16.783392 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6436a7d43dab46bca948438f832245ed" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:16.783475 31626 raft_consensus.cc:2454] T 6436a7d43dab46bca948438f832245ed P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:16.783551 31888 raft_consensus.cc:2454] T 6436a7d43dab46bca948438f832245ed P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:16.783777 31691 leader_election.cc:302] T 6436a7d43dab46bca948438f832245ed P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:16.783932 692 raft_consensus.cc:2781] T 6436a7d43dab46bca948438f832245ed P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:16.783989 692 raft_consensus.cc:482] T 6436a7d43dab46bca948438f832245ed P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:16.784034 692 raft_consensus.cc:3037] T 6436a7d43dab46bca948438f832245ed P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:16.784832 692 raft_consensus.cc:504] T 6436a7d43dab46bca948438f832245ed P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:16.785156 692 leader_election.cc:288] T 6436a7d43dab46bca948438f832245ed P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:16.785481 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6436a7d43dab46bca948438f832245ed" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:16.785605 31626 raft_consensus.cc:3037] T 6436a7d43dab46bca948438f832245ed P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:16.785576 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6436a7d43dab46bca948438f832245ed" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:16.785691 31888 raft_consensus.cc:3037] T 6436a7d43dab46bca948438f832245ed P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:16.786521 31626 raft_consensus.cc:2454] T 6436a7d43dab46bca948438f832245ed P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:16.786773 31888 raft_consensus.cc:2454] T 6436a7d43dab46bca948438f832245ed P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:16.786808 31691 leader_election.cc:302] T 6436a7d43dab46bca948438f832245ed P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:16.786955 692 raft_consensus.cc:2781] T 6436a7d43dab46bca948438f832245ed P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:16.787045 692 raft_consensus.cc:686] T 6436a7d43dab46bca948438f832245ed P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Becoming Leader. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Running, Role: LEADER
I20240502 20:46:16.787194 692 consensus_queue.cc:237] T 6436a7d43dab46bca948438f832245ed P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:16.788933 29036 catalog_manager.cc:5484] T 6436a7d43dab46bca948438f832245ed P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: term changed from 0 to 1, leader changed from <none> to e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71). New cstate: current_term: 1 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:16.811672 29293 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:16.812089 31602 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:16.814779 31474 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:16.817298 29426 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:16.820766 31867 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:16.821938 29162 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:16.843406 29036 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:16.846388 31736 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:16.848886 29557 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:16.850889 31602 tablet_service.cc:1467] Processing CreateTablet for tablet a4188bca839b4b238e05aaf78c7e58b0 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d38f9ea0a0b6419abf445830750f9f47]), partition=
I20240502 20:46:16.851075 31474 tablet_service.cc:1467] Processing CreateTablet for tablet a4188bca839b4b238e05aaf78c7e58b0 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d38f9ea0a0b6419abf445830750f9f47]), partition=
I20240502 20:46:16.851181 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4188bca839b4b238e05aaf78c7e58b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:16.851320 31474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4188bca839b4b238e05aaf78c7e58b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:16.853309 1151 tablet_bootstrap.cc:492] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:16.853441 1154 tablet_bootstrap.cc:492] T a4188bca839b4b238e05aaf78c7e58b0 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:16.854154 1151 tablet_bootstrap.cc:654] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:16.854321 1154 tablet_bootstrap.cc:654] T a4188bca839b4b238e05aaf78c7e58b0 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:16.855058 31867 tablet_service.cc:1467] Processing CreateTablet for tablet a4188bca839b4b238e05aaf78c7e58b0 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d38f9ea0a0b6419abf445830750f9f47]), partition=
I20240502 20:46:16.855284 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4188bca839b4b238e05aaf78c7e58b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:16.857152 681 tablet_bootstrap.cc:492] T a4188bca839b4b238e05aaf78c7e58b0 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:16.858242 681 tablet_bootstrap.cc:654] T a4188bca839b4b238e05aaf78c7e58b0 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:16.868194 1154 tablet_bootstrap.cc:492] T a4188bca839b4b238e05aaf78c7e58b0 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:16.868330 1154 ts_tablet_manager.cc:1367] T a4188bca839b4b238e05aaf78c7e58b0 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 20:46:16.868870 1154 raft_consensus.cc:348] T a4188bca839b4b238e05aaf78c7e58b0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.868989 1154 raft_consensus.cc:374] T a4188bca839b4b238e05aaf78c7e58b0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:16.869014 1154 raft_consensus.cc:729] T a4188bca839b4b238e05aaf78c7e58b0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:16.869124 1154 consensus_queue.cc:260] T a4188bca839b4b238e05aaf78c7e58b0 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.869388 1154 ts_tablet_manager.cc:1398] T a4188bca839b4b238e05aaf78c7e58b0 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:16.882925 681 tablet_bootstrap.cc:492] T a4188bca839b4b238e05aaf78c7e58b0 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:16.883040 681 ts_tablet_manager.cc:1367] T a4188bca839b4b238e05aaf78c7e58b0 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.000s
I20240502 20:46:16.883458 681 raft_consensus.cc:348] T a4188bca839b4b238e05aaf78c7e58b0 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.883550 681 raft_consensus.cc:374] T a4188bca839b4b238e05aaf78c7e58b0 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:16.883570 681 raft_consensus.cc:729] T a4188bca839b4b238e05aaf78c7e58b0 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:16.883682 681 consensus_queue.cc:260] T a4188bca839b4b238e05aaf78c7e58b0 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.883936 681 ts_tablet_manager.cc:1398] T a4188bca839b4b238e05aaf78c7e58b0 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:16.884438 1151 tablet_bootstrap.cc:492] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:16.884541 1151 ts_tablet_manager.cc:1367] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.031s user 0.000s sys 0.002s
I20240502 20:46:16.885003 1151 raft_consensus.cc:348] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.885118 1151 raft_consensus.cc:374] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:16.885149 1151 raft_consensus.cc:729] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:16.885285 1151 consensus_queue.cc:260] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.885563 1151 ts_tablet_manager.cc:1398] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:16.932233 31888 consensus_queue.cc:237] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:16.933686 31494 raft_consensus.cc:1261] T 0f57649f122e45f9b8ef82ec975e5359 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:16.933687 31626 raft_consensus.cc:1261] T 0f57649f122e45f9b8ef82ec975e5359 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 20:46:16.933971 31823 consensus_peers.cc:485] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691 -> Peer 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): Couldn't send request to peer 09258ac095664fa99caf0707262cde13. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0f57649f122e45f9b8ef82ec975e5359. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:16.934118 1483 consensus_queue.cc:1035] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:16.934315 32172 consensus_queue.cc:1035] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:16.936254 32172 raft_consensus.cc:2932] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:16.936967 31626 raft_consensus.cc:2932] T 0f57649f122e45f9b8ef82ec975e5359 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:16.938726 31494 raft_consensus.cc:2932] T 0f57649f122e45f9b8ef82ec975e5359 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:16.939921 29033 catalog_manager.cc:5484] T 0f57649f122e45f9b8ef82ec975e5359 P 315915efcf2b413ca7f9d874338e1b2d reported cstate change: config changed from index -1 to 2, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) added. New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:16.958067 29446 consensus_queue.cc:237] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:16.959465 29577 raft_consensus.cc:1261] T 4c4440d27d9e4070a8e425d26d8a0772 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:16.959465 29175 raft_consensus.cc:1261] T 4c4440d27d9e4070a8e425d26d8a0772 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 20:46:16.959829 29380 consensus_peers.cc:485] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): Couldn't send request to peer e6b60803d0764d228c863e2da65e3691. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4c4440d27d9e4070a8e425d26d8a0772. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:16.959861 1503 consensus_queue.cc:1035] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:16.960037 1320 consensus_queue.cc:1035] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:16.961803 1320 raft_consensus.cc:2932] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:16.962251 29175 raft_consensus.cc:2932] T 4c4440d27d9e4070a8e425d26d8a0772 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:16.963600 29577 raft_consensus.cc:2932] T 4c4440d27d9e4070a8e425d26d8a0772 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:16.964965 29036 catalog_manager.cc:5484] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:16.982686 29315 consensus_queue.cc:237] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.984115 29446 raft_consensus.cc:1261] T 741da32f7c924490a5fa5933708adf30 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:16.984220 29577 raft_consensus.cc:1261] T 741da32f7c924490a5fa5933708adf30 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 20:46:16.984234 29250 consensus_peers.cc:485] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13 -> Peer 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): Couldn't send request to peer 2a9c1af8cf814030bde56b67ca5c2fbe. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 741da32f7c924490a5fa5933708adf30. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:16.984555 1408 consensus_queue.cc:1035] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:16.984757 830 consensus_queue.cc:1035] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:16.986893 1408 raft_consensus.cc:2932] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:16.987321 29446 raft_consensus.cc:2932] T 741da32f7c924490a5fa5933708adf30 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:16.989147 29577 raft_consensus.cc:2932] T 741da32f7c924490a5fa5933708adf30 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:16.992216 29039 catalog_manager.cc:5484] T 741da32f7c924490a5fa5933708adf30 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:16.998579 1055 raft_consensus.cc:482] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:16.998696 1055 raft_consensus.cc:504] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:16.999126 1055 leader_election.cc:288] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:16.999516 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4188bca839b4b238e05aaf78c7e58b0" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:16.999660 31494 raft_consensus.cc:2454] T a4188bca839b4b238e05aaf78c7e58b0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:17.000061 31561 leader_election.cc:302] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, 315915efcf2b413ca7f9d874338e1b2d; no voters:
I20240502 20:46:17.000231 1055 raft_consensus.cc:2781] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:17.000273 1055 raft_consensus.cc:482] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:17.000305 1055 raft_consensus.cc:3037] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:17.000504 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4188bca839b4b238e05aaf78c7e58b0" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:17.000633 31888 raft_consensus.cc:2454] T a4188bca839b4b238e05aaf78c7e58b0 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:17.001302 1055 raft_consensus.cc:504] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:17.001657 1055 leader_election.cc:288] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Requested vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:17.001878 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4188bca839b4b238e05aaf78c7e58b0" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:17.001955 31888 raft_consensus.cc:3037] T a4188bca839b4b238e05aaf78c7e58b0 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:17.002887 31888 raft_consensus.cc:2454] T a4188bca839b4b238e05aaf78c7e58b0 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:17.003156 31560 leader_election.cc:302] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:17.003343 1055 raft_consensus.cc:2781] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:17.003410 1055 raft_consensus.cc:686] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Becoming Leader. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Running, Role: LEADER
I20240502 20:46:17.003543 1055 consensus_queue.cc:237] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:17.005107 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4188bca839b4b238e05aaf78c7e58b0" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:17.005229 31494 raft_consensus.cc:3037] T a4188bca839b4b238e05aaf78c7e58b0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:17.005913 29036 catalog_manager.cc:5484] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70). New cstate: current_term: 1 leader_uuid: "315915efcf2b413ca7f9d874338e1b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:17.006155 31494 raft_consensus.cc:2454] T a4188bca839b4b238e05aaf78c7e58b0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:17.015683 31888 consensus_queue.cc:237] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:17.016911 31626 raft_consensus.cc:1261] T 11ee3029cd744581ba9c5c213ca8fd2b P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 20:46:17.017251 31823 consensus_peers.cc:485] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 -> Peer 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): Couldn't send request to peer 09258ac095664fa99caf0707262cde13. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 11ee3029cd744581ba9c5c213ca8fd2b. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:17.017297 1486 consensus_queue.cc:1035] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:17.017328 31757 raft_consensus.cc:1261] T 11ee3029cd744581ba9c5c213ca8fd2b P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:17.017692 1486 consensus_queue.cc:1035] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:17.019294 1483 raft_consensus.cc:2932] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:17.019726 31757 raft_consensus.cc:2932] T 11ee3029cd744581ba9c5c213ca8fd2b P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:17.021188 31626 raft_consensus.cc:2932] T 11ee3029cd744581ba9c5c213ca8fd2b P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:17.022763 29033 catalog_manager.cc:5484] T 11ee3029cd744581ba9c5c213ca8fd2b P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: config changed from index -1 to 2, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) added. New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:17.101929 29293 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:17.102509 29162 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:17.104878 31602 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:17.111344 31867 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:17.118744 31474 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:17.122819 29557 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:17.127694 29038 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:17.137122 31736 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:17.137637 31494 raft_consensus.cc:1261] T a4188bca839b4b238e05aaf78c7e58b0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Refusing update from remote peer 315915efcf2b413ca7f9d874338e1b2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:17.138196 1379 consensus_queue.cc:1035] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:17.140707 29038 catalog_manager.cc:5576] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887) does not have the latest schema for tablet a4188bca839b4b238e05aaf78c7e58b0 (table rebalancer_dynamic_table [id=d38f9ea0a0b6419abf445830750f9f47]). Expected version 1 got 0
I20240502 20:46:17.147984 29038 catalog_manager.cc:5576] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887) does not have the latest schema for tablet a4188bca839b4b238e05aaf78c7e58b0 (table rebalancer_dynamic_table [id=d38f9ea0a0b6419abf445830750f9f47]). Expected version 1 got 0
I20240502 20:46:17.149209 1055 consensus_queue.cc:1035] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:17.149530 1596 tablet.cc:1722] T a4188bca839b4b238e05aaf78c7e58b0 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:17.150218 29426 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:17.152400 1596 tablet.cc:1620] T a4188bca839b4b238e05aaf78c7e58b0 P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:17.155138 1055 consensus_queue.cc:1035] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:17.160143 29041 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=d38f9ea0a0b6419abf445830750f9f47] alter complete (version 1)
I20240502 20:46:17.161193 31888 raft_consensus.cc:1261] T a4188bca839b4b238e05aaf78c7e58b0 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer 315915efcf2b413ca7f9d874338e1b2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:17.161785 1598 consensus_queue.cc:1035] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:17.163341 1604 tablet.cc:1722] T a4188bca839b4b238e05aaf78c7e58b0 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:17.163985 1604 tablet.cc:1620] T a4188bca839b4b238e05aaf78c7e58b0 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:17.164729 29041 catalog_manager.cc:5576] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207) does not have the latest schema for tablet a4188bca839b4b238e05aaf78c7e58b0 (table rebalancer_dynamic_table [id=d38f9ea0a0b6419abf445830750f9f47]). Expected version 1 got 0
I20240502 20:46:17.167465 1595 tablet.cc:1722] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:17.167804 1595 tablet.cc:1620] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:17.170845 692 consensus_queue.cc:1035] T 82f57a76ce934ebab6c68948c543e0cb P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:17.180431 975 consensus_queue.cc:1035] T 82f57a76ce934ebab6c68948c543e0cb P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:17.212817 692 consensus_queue.cc:1035] T 6436a7d43dab46bca948438f832245ed P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:17.216648 975 consensus_queue.cc:1035] T 6436a7d43dab46bca948438f832245ed P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:17.262759 31474 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:17.264590 31602 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:17.265182 29293 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:17.265949 29037 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:17.271968 1595 tablet.cc:1722] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:17.272316 1595 tablet.cc:1620] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:17.273638 29162 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:17.275632 1604 tablet.cc:1722] T a4188bca839b4b238e05aaf78c7e58b0 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:17.275938 1604 tablet.cc:1620] T a4188bca839b4b238e05aaf78c7e58b0 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:17.277623 1596 tablet.cc:1722] T a4188bca839b4b238e05aaf78c7e58b0 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:17.277909 1596 tablet.cc:1620] T a4188bca839b4b238e05aaf78c7e58b0 P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:17.278956 29426 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:17.286794 29036 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=d38f9ea0a0b6419abf445830750f9f47] alter complete (version 2)
I20240502 20:46:17.291718 31867 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:17.300406 31736 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:17.301126 29557 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:17.340044 29037 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_14"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:17.346657 31602 tablet_service.cc:1467] Processing CreateTablet for tablet 2e4a0e5ef89d447b96b998f085e3495c (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=125bd901c1064f21903d5bf6eefae338]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:17.346995 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e4a0e5ef89d447b96b998f085e3495c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:17.347481 31866 tablet_service.cc:1467] Processing CreateTablet for tablet a58b3a6ce8c042c8bd371d52e1f2a270 (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=125bd901c1064f21903d5bf6eefae338]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:17.347487 31474 tablet_service.cc:1467] Processing CreateTablet for tablet 2e4a0e5ef89d447b96b998f085e3495c (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=125bd901c1064f21903d5bf6eefae338]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:17.347769 31474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e4a0e5ef89d447b96b998f085e3495c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:17.347769 31866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a58b3a6ce8c042c8bd371d52e1f2a270. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:17.347767 31736 tablet_service.cc:1467] Processing CreateTablet for tablet 7db3d66689ef4fb2869e94448515ad12 (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=125bd901c1064f21903d5bf6eefae338]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:17.348026 31736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7db3d66689ef4fb2869e94448515ad12. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:17.350003 31868 tablet_service.cc:1467] Processing CreateTablet for tablet 7db3d66689ef4fb2869e94448515ad12 (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=125bd901c1064f21903d5bf6eefae338]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:17.350003 31475 tablet_service.cc:1467] Processing CreateTablet for tablet 7db3d66689ef4fb2869e94448515ad12 (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=125bd901c1064f21903d5bf6eefae338]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:17.350229 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7db3d66689ef4fb2869e94448515ad12. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:17.350229 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7db3d66689ef4fb2869e94448515ad12. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:17.350355 31735 tablet_service.cc:1467] Processing CreateTablet for tablet a58b3a6ce8c042c8bd371d52e1f2a270 (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=125bd901c1064f21903d5bf6eefae338]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:17.350615 31735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a58b3a6ce8c042c8bd371d52e1f2a270. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:17.350615 1151 tablet_bootstrap.cc:492] T 2e4a0e5ef89d447b96b998f085e3495c P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:17.351565 1151 tablet_bootstrap.cc:654] T 2e4a0e5ef89d447b96b998f085e3495c P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:17.352151 681 tablet_bootstrap.cc:492] T a58b3a6ce8c042c8bd371d52e1f2a270 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:17.352344 29426 tablet_service.cc:1467] Processing CreateTablet for tablet a58b3a6ce8c042c8bd371d52e1f2a270 (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=125bd901c1064f21903d5bf6eefae338]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:17.347501 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 2e4a0e5ef89d447b96b998f085e3495c (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=125bd901c1064f21903d5bf6eefae338]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:17.352721 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e4a0e5ef89d447b96b998f085e3495c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:17.352723 29426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a58b3a6ce8c042c8bd371d52e1f2a270. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:17.352996 1151 tablet_bootstrap.cc:492] T 2e4a0e5ef89d447b96b998f085e3495c P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:17.353073 1151 ts_tablet_manager.cc:1367] T 2e4a0e5ef89d447b96b998f085e3495c P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 20:46:17.353155 681 tablet_bootstrap.cc:654] T a58b3a6ce8c042c8bd371d52e1f2a270 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:17.353495 1151 raft_consensus.cc:348] T 2e4a0e5ef89d447b96b998f085e3495c P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:17.353602 1151 raft_consensus.cc:374] T 2e4a0e5ef89d447b96b998f085e3495c P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:17.353638 1151 raft_consensus.cc:729] T 2e4a0e5ef89d447b96b998f085e3495c P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:17.353771 1151 consensus_queue.cc:260] T 2e4a0e5ef89d447b96b998f085e3495c P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:17.354065 1151 ts_tablet_manager.cc:1398] T 2e4a0e5ef89d447b96b998f085e3495c P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:17.354924 1670 tablet_bootstrap.cc:492] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:17.356132 1670 tablet_bootstrap.cc:654] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:17.358111 1670 tablet_bootstrap.cc:492] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:17.358208 1670 ts_tablet_manager.cc:1367] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:17.358669 1670 raft_consensus.cc:348] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:17.358858 1670 raft_consensus.cc:374] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:17.358893 1670 raft_consensus.cc:729] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:17.358943 1154 tablet_bootstrap.cc:492] T 2e4a0e5ef89d447b96b998f085e3495c P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:17.359102 1670 consensus_queue.cc:260] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:17.359911 1154 tablet_bootstrap.cc:654] T 2e4a0e5ef89d447b96b998f085e3495c P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:17.361225 1670 ts_tablet_manager.cc:1398] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:17.362154 681 tablet_bootstrap.cc:492] T a58b3a6ce8c042c8bd371d52e1f2a270 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:17.362267 681 ts_tablet_manager.cc:1367] T a58b3a6ce8c042c8bd371d52e1f2a270 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 20:46:17.362907 1670 tablet_bootstrap.cc:492] T a58b3a6ce8c042c8bd371d52e1f2a270 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:17.362843 681 raft_consensus.cc:348] T a58b3a6ce8c042c8bd371d52e1f2a270 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:17.362958 681 raft_consensus.cc:374] T a58b3a6ce8c042c8bd371d52e1f2a270 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:17.362988 681 raft_consensus.cc:729] T a58b3a6ce8c042c8bd371d52e1f2a270 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:17.363137 681 consensus_queue.cc:260] T a58b3a6ce8c042c8bd371d52e1f2a270 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:17.363416 681 ts_tablet_manager.cc:1398] T a58b3a6ce8c042c8bd371d52e1f2a270 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:17.363600 681 tablet_bootstrap.cc:492] T 7db3d66689ef4fb2869e94448515ad12 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:17.363948 1670 tablet_bootstrap.cc:654] T a58b3a6ce8c042c8bd371d52e1f2a270 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:17.364517 681 tablet_bootstrap.cc:654] T 7db3d66689ef4fb2869e94448515ad12 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:17.364928 1154 tablet_bootstrap.cc:492] T 2e4a0e5ef89d447b96b998f085e3495c P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:17.364995 1154 ts_tablet_manager.cc:1367] T 2e4a0e5ef89d447b96b998f085e3495c P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 20:46:17.365406 1154 raft_consensus.cc:348] T 2e4a0e5ef89d447b96b998f085e3495c P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:17.365489 1154 raft_consensus.cc:374] T 2e4a0e5ef89d447b96b998f085e3495c P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:17.365509 1154 raft_consensus.cc:729] T 2e4a0e5ef89d447b96b998f085e3495c P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:17.365610 1154 consensus_queue.cc:260] T 2e4a0e5ef89d447b96b998f085e3495c P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:17.365870 1154 ts_tablet_manager.cc:1398] T 2e4a0e5ef89d447b96b998f085e3495c P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:17.366014 1154 tablet_bootstrap.cc:492] T 7db3d66689ef4fb2869e94448515ad12 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:17.366820 1154 tablet_bootstrap.cc:654] T 7db3d66689ef4fb2869e94448515ad12 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:17.367697 1154 tablet_bootstrap.cc:492] T 7db3d66689ef4fb2869e94448515ad12 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:17.367767 1154 ts_tablet_manager.cc:1367] T 7db3d66689ef4fb2869e94448515ad12 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:17.368919 1154 raft_consensus.cc:348] T 7db3d66689ef4fb2869e94448515ad12 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:17.369014 1154 raft_consensus.cc:374] T 7db3d66689ef4fb2869e94448515ad12 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:17.369043 1154 raft_consensus.cc:729] T 7db3d66689ef4fb2869e94448515ad12 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:17.369153 1154 consensus_queue.cc:260] T 7db3d66689ef4fb2869e94448515ad12 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:17.369485 1154 ts_tablet_manager.cc:1398] T 7db3d66689ef4fb2869e94448515ad12 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:17.370867 1670 tablet_bootstrap.cc:492] T a58b3a6ce8c042c8bd371d52e1f2a270 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:17.370963 1670 ts_tablet_manager.cc:1367] T a58b3a6ce8c042c8bd371d52e1f2a270 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240502 20:46:17.371510 1670 raft_consensus.cc:348] T a58b3a6ce8c042c8bd371d52e1f2a270 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:17.371696 1670 raft_consensus.cc:374] T a58b3a6ce8c042c8bd371d52e1f2a270 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:17.371941 1670 raft_consensus.cc:729] T a58b3a6ce8c042c8bd371d52e1f2a270 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:17.372118 1670 consensus_queue.cc:260] T a58b3a6ce8c042c8bd371d52e1f2a270 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:17.372478 1670 ts_tablet_manager.cc:1398] T a58b3a6ce8c042c8bd371d52e1f2a270 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:17.373317 681 tablet_bootstrap.cc:492] T 7db3d66689ef4fb2869e94448515ad12 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:17.373464 681 ts_tablet_manager.cc:1367] T 7db3d66689ef4fb2869e94448515ad12 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 20:46:17.373970 681 raft_consensus.cc:348] T 7db3d66689ef4fb2869e94448515ad12 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:17.374140 681 raft_consensus.cc:374] T 7db3d66689ef4fb2869e94448515ad12 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:17.374192 681 raft_consensus.cc:729] T 7db3d66689ef4fb2869e94448515ad12 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:17.374339 681 consensus_queue.cc:260] T 7db3d66689ef4fb2869e94448515ad12 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:17.374697 681 ts_tablet_manager.cc:1398] T 7db3d66689ef4fb2869e94448515ad12 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:17.375154 681 tablet_bootstrap.cc:492] T 2e4a0e5ef89d447b96b998f085e3495c P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:17.376114 681 tablet_bootstrap.cc:654] T 2e4a0e5ef89d447b96b998f085e3495c P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:17.382720 1675 tablet_bootstrap.cc:492] T a58b3a6ce8c042c8bd371d52e1f2a270 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:17.383865 1675 tablet_bootstrap.cc:654] T a58b3a6ce8c042c8bd371d52e1f2a270 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:17.385217 1675 tablet_bootstrap.cc:492] T a58b3a6ce8c042c8bd371d52e1f2a270 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:17.385354 1675 ts_tablet_manager.cc:1367] T a58b3a6ce8c042c8bd371d52e1f2a270 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:17.385856 1675 raft_consensus.cc:348] T a58b3a6ce8c042c8bd371d52e1f2a270 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:17.386035 1675 raft_consensus.cc:374] T a58b3a6ce8c042c8bd371d52e1f2a270 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:17.386111 1675 raft_consensus.cc:729] T a58b3a6ce8c042c8bd371d52e1f2a270 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:17.386277 1675 consensus_queue.cc:260] T a58b3a6ce8c042c8bd371d52e1f2a270 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:17.386658 1675 ts_tablet_manager.cc:1398] T a58b3a6ce8c042c8bd371d52e1f2a270 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:17.392311 681 tablet_bootstrap.cc:492] T 2e4a0e5ef89d447b96b998f085e3495c P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:17.392419 681 ts_tablet_manager.cc:1367] T 2e4a0e5ef89d447b96b998f085e3495c P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 20:46:17.392856 681 raft_consensus.cc:348] T 2e4a0e5ef89d447b96b998f085e3495c P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:17.392973 681 raft_consensus.cc:374] T 2e4a0e5ef89d447b96b998f085e3495c P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:17.392998 681 raft_consensus.cc:729] T 2e4a0e5ef89d447b96b998f085e3495c P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:17.393143 681 consensus_queue.cc:260] T 2e4a0e5ef89d447b96b998f085e3495c P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:17.393478 681 ts_tablet_manager.cc:1398] T 2e4a0e5ef89d447b96b998f085e3495c P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:17.394344 29037 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:17.401053 1595 tablet.cc:1722] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:17.401436 1596 tablet.cc:1722] T a4188bca839b4b238e05aaf78c7e58b0 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:17.402298 1604 tablet.cc:1722] T a4188bca839b4b238e05aaf78c7e58b0 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:17.405984 1486 raft_consensus.cc:482] T a58b3a6ce8c042c8bd371d52e1f2a270 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:17.406072 1486 raft_consensus.cc:504] T a58b3a6ce8c042c8bd371d52e1f2a270 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:17.406595 1486 leader_election.cc:288] T a58b3a6ce8c042c8bd371d52e1f2a270 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:17.406951 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a58b3a6ce8c042c8bd371d52e1f2a270" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:17.407092 29446 raft_consensus.cc:2454] T a58b3a6ce8c042c8bd371d52e1f2a270 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:17.407063 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a58b3a6ce8c042c8bd371d52e1f2a270" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:17.407204 31757 raft_consensus.cc:2454] T a58b3a6ce8c042c8bd371d52e1f2a270 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:17.407426 31823 leader_election.cc:302] T a58b3a6ce8c042c8bd371d52e1f2a270 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dbcdc6ac635b44b185c3eba63f66e9d6, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:17.407521 29038 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=d38f9ea0a0b6419abf445830750f9f47] alter complete (version 3)
I20240502 20:46:17.407626 1486 raft_consensus.cc:2781] T a58b3a6ce8c042c8bd371d52e1f2a270 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:17.407665 1486 raft_consensus.cc:482] T a58b3a6ce8c042c8bd371d52e1f2a270 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:17.407688 1486 raft_consensus.cc:3037] T a58b3a6ce8c042c8bd371d52e1f2a270 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:17.408489 1486 raft_consensus.cc:504] T a58b3a6ce8c042c8bd371d52e1f2a270 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:17.408811 1486 leader_election.cc:288] T a58b3a6ce8c042c8bd371d52e1f2a270 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Requested vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:17.409122 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a58b3a6ce8c042c8bd371d52e1f2a270" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:17.409242 29446 raft_consensus.cc:3037] T a58b3a6ce8c042c8bd371d52e1f2a270 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:17.409235 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a58b3a6ce8c042c8bd371d52e1f2a270" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:17.409354 31757 raft_consensus.cc:3037] T a58b3a6ce8c042c8bd371d52e1f2a270 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:17.410063 31757 raft_consensus.cc:2454] T a58b3a6ce8c042c8bd371d52e1f2a270 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:17.410183 29446 raft_consensus.cc:2454] T a58b3a6ce8c042c8bd371d52e1f2a270 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:17.410359 31821 leader_election.cc:302] T a58b3a6ce8c042c8bd371d52e1f2a270 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e6b60803d0764d228c863e2da65e3691, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:17.410692 1486 raft_consensus.cc:2781] T a58b3a6ce8c042c8bd371d52e1f2a270 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:17.410789 1486 raft_consensus.cc:686] T a58b3a6ce8c042c8bd371d52e1f2a270 P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Becoming Leader. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Running, Role: LEADER
I20240502 20:46:17.410915 1486 consensus_queue.cc:237] T a58b3a6ce8c042c8bd371d52e1f2a270 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:17.412611 29037 catalog_manager.cc:5484] T a58b3a6ce8c042c8bd371d52e1f2a270 P e6b60803d0764d228c863e2da65e3691 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6b60803d0764d228c863e2da65e3691 (127.22.45.72). New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:17.425304 1694 ts_tablet_manager.cc:902] T 0f57649f122e45f9b8ef82ec975e5359 P 09258ac095664fa99caf0707262cde13: Initiating tablet copy from peer e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:17.426152 1694 tablet_copy_client.cc:308] T 0f57649f122e45f9b8ef82ec975e5359 P 09258ac095664fa99caf0707262cde13: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.72:35207
I20240502 20:46:17.432277 31908 tablet_copy_service.cc:140] P e6b60803d0764d228c863e2da65e3691: Received BeginTabletCopySession request for tablet 0f57649f122e45f9b8ef82ec975e5359 from peer 09258ac095664fa99caf0707262cde13 ({username='slave'} at 127.22.45.66:55365)
I20240502 20:46:17.432431 31908 tablet_copy_service.cc:161] P e6b60803d0764d228c863e2da65e3691: Beginning new tablet copy session on tablet 0f57649f122e45f9b8ef82ec975e5359 from peer 09258ac095664fa99caf0707262cde13 at {username='slave'} at 127.22.45.66:55365: session id = 09258ac095664fa99caf0707262cde13-0f57649f122e45f9b8ef82ec975e5359
I20240502 20:46:17.433517 31908 tablet_copy_source_session.cc:205] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:17.434963 1694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f57649f122e45f9b8ef82ec975e5359. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:17.437397 1694 tablet_copy_client.cc:783] T 0f57649f122e45f9b8ef82ec975e5359 P 09258ac095664fa99caf0707262cde13: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:17.437582 1694 tablet_copy_client.cc:647] T 0f57649f122e45f9b8ef82ec975e5359 P 09258ac095664fa99caf0707262cde13: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:17.439292 1694 tablet_copy_client.cc:523] T 0f57649f122e45f9b8ef82ec975e5359 P 09258ac095664fa99caf0707262cde13: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:17.440721 1694 tablet_bootstrap.cc:492] T 0f57649f122e45f9b8ef82ec975e5359 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:17.448515 1694 tablet_bootstrap.cc:492] T 0f57649f122e45f9b8ef82ec975e5359 P 09258ac095664fa99caf0707262cde13: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:17.448819 1694 tablet_bootstrap.cc:492] T 0f57649f122e45f9b8ef82ec975e5359 P 09258ac095664fa99caf0707262cde13: Bootstrap complete.
I20240502 20:46:17.448998 1694 ts_tablet_manager.cc:1367] T 0f57649f122e45f9b8ef82ec975e5359 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20240502 20:46:17.449491 1694 raft_consensus.cc:348] T 0f57649f122e45f9b8ef82ec975e5359 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:17.449626 1694 raft_consensus.cc:729] T 0f57649f122e45f9b8ef82ec975e5359 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:17.449766 1694 consensus_queue.cc:260] T 0f57649f122e45f9b8ef82ec975e5359 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:17.450066 1694 ts_tablet_manager.cc:1398] T 0f57649f122e45f9b8ef82ec975e5359 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:17.451300 31908 tablet_copy_service.cc:342] P e6b60803d0764d228c863e2da65e3691: Request end of tablet copy session 09258ac095664fa99caf0707262cde13-0f57649f122e45f9b8ef82ec975e5359 received from {username='slave'} at 127.22.45.66:55365
I20240502 20:46:17.451398 31908 tablet_copy_service.cc:433] P e6b60803d0764d228c863e2da65e3691: ending tablet copy session 09258ac095664fa99caf0707262cde13-0f57649f122e45f9b8ef82ec975e5359 on tablet 0f57649f122e45f9b8ef82ec975e5359 with peer 09258ac095664fa99caf0707262cde13
I20240502 20:46:17.496420 1720 ts_tablet_manager.cc:902] T 741da32f7c924490a5fa5933708adf30 P 2a9c1af8cf814030bde56b67ca5c2fbe: Initiating tablet copy from peer 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:17.496939 1720 tablet_copy_client.cc:308] T 741da32f7c924490a5fa5933708adf30 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.66:40945
I20240502 20:46:17.497445 29335 tablet_copy_service.cc:140] P 09258ac095664fa99caf0707262cde13: Received BeginTabletCopySession request for tablet 741da32f7c924490a5fa5933708adf30 from peer 2a9c1af8cf814030bde56b67ca5c2fbe ({username='slave'} at 127.22.45.69:52445)
I20240502 20:46:17.497547 29335 tablet_copy_service.cc:161] P 09258ac095664fa99caf0707262cde13: Beginning new tablet copy session on tablet 741da32f7c924490a5fa5933708adf30 from peer 2a9c1af8cf814030bde56b67ca5c2fbe at {username='slave'} at 127.22.45.69:52445: session id = 2a9c1af8cf814030bde56b67ca5c2fbe-741da32f7c924490a5fa5933708adf30
I20240502 20:46:17.498323 29335 tablet_copy_source_session.cc:205] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:17.499207 1720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 741da32f7c924490a5fa5933708adf30. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:17.501019 1720 tablet_copy_client.cc:783] T 741da32f7c924490a5fa5933708adf30 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:17.501176 1720 tablet_copy_client.cc:647] T 741da32f7c924490a5fa5933708adf30 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:17.502276 1720 tablet_copy_client.cc:523] T 741da32f7c924490a5fa5933708adf30 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:17.503412 1720 tablet_bootstrap.cc:492] T 741da32f7c924490a5fa5933708adf30 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:17.507337 977 raft_consensus.cc:482] T 2e4a0e5ef89d447b96b998f085e3495c P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:17.507426 977 raft_consensus.cc:504] T 2e4a0e5ef89d447b96b998f085e3495c P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:17.507790 977 leader_election.cc:288] T 2e4a0e5ef89d447b96b998f085e3495c P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:17.508266 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e4a0e5ef89d447b96b998f085e3495c" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:17.508432 31888 raft_consensus.cc:2454] T 2e4a0e5ef89d447b96b998f085e3495c P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:17.508734 31429 leader_election.cc:302] T 2e4a0e5ef89d447b96b998f085e3495c P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:17.508739 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e4a0e5ef89d447b96b998f085e3495c" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:17.508890 31626 raft_consensus.cc:2454] T 2e4a0e5ef89d447b96b998f085e3495c P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:17.508899 977 raft_consensus.cc:2781] T 2e4a0e5ef89d447b96b998f085e3495c P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:17.508944 977 raft_consensus.cc:482] T 2e4a0e5ef89d447b96b998f085e3495c P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:17.508970 977 raft_consensus.cc:3037] T 2e4a0e5ef89d447b96b998f085e3495c P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:17.509758 977 raft_consensus.cc:504] T 2e4a0e5ef89d447b96b998f085e3495c P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:17.510039 977 leader_election.cc:288] T 2e4a0e5ef89d447b96b998f085e3495c P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:17.510550 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e4a0e5ef89d447b96b998f085e3495c" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:17.510568 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e4a0e5ef89d447b96b998f085e3495c" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:17.510671 31888 raft_consensus.cc:3037] T 2e4a0e5ef89d447b96b998f085e3495c P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:17.510720 31626 raft_consensus.cc:3037] T 2e4a0e5ef89d447b96b998f085e3495c P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:17.511575 31888 raft_consensus.cc:2454] T 2e4a0e5ef89d447b96b998f085e3495c P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:17.511756 31626 raft_consensus.cc:2454] T 2e4a0e5ef89d447b96b998f085e3495c P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:17.511847 31429 leader_election.cc:302] T 2e4a0e5ef89d447b96b998f085e3495c P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:17.512373 29039 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:17.512513 29039 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:17.512928 977 raft_consensus.cc:2781] T 2e4a0e5ef89d447b96b998f085e3495c P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:17.513015 977 raft_consensus.cc:686] T 2e4a0e5ef89d447b96b998f085e3495c P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Becoming Leader. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Running, Role: LEADER
I20240502 20:46:17.513140 977 consensus_queue.cc:237] T 2e4a0e5ef89d447b96b998f085e3495c P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:17.515303 29039 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet a4188bca839b4b238e05aaf78c7e58b0
I20240502 20:46:17.515518 29037 catalog_manager.cc:5484] T 2e4a0e5ef89d447b96b998f085e3495c P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69). New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:17.516203 31602 tablet_service.cc:1514] Processing DeleteTablet for tablet a4188bca839b4b238e05aaf78c7e58b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:17 UTC) from {username='slave'} at 127.0.0.1:58346
I20240502 20:46:17.516207 31867 tablet_service.cc:1514] Processing DeleteTablet for tablet a4188bca839b4b238e05aaf78c7e58b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:17 UTC) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:17.516498 1727 tablet_replica.cc:330] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d: stopping tablet replica
I20240502 20:46:17.516644 1727 raft_consensus.cc:2229] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:17.516757 31475 tablet_service.cc:1514] Processing DeleteTablet for tablet a4188bca839b4b238e05aaf78c7e58b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:17 UTC) from {username='slave'} at 127.0.0.1:40276
I20240502 20:46:17.517063 1729 tablet_replica.cc:330] T a4188bca839b4b238e05aaf78c7e58b0 P 2a9c1af8cf814030bde56b67ca5c2fbe: stopping tablet replica
I20240502 20:46:17.517180 1727 raft_consensus.cc:2258] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:17.517194 1729 raft_consensus.cc:2229] T a4188bca839b4b238e05aaf78c7e58b0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:17.517306 1729 raft_consensus.cc:2258] T a4188bca839b4b238e05aaf78c7e58b0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:17.518034 1729 ts_tablet_manager.cc:1875] T a4188bca839b4b238e05aaf78c7e58b0 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:17.519232 1727 ts_tablet_manager.cc:1875] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:17.520879 1729 ts_tablet_manager.cc:1888] T a4188bca839b4b238e05aaf78c7e58b0 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:17.521055 1729 log.cc:1196] T a4188bca839b4b238e05aaf78c7e58b0 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/wals/a4188bca839b4b238e05aaf78c7e58b0
I20240502 20:46:17.521536 1729 ts_tablet_manager.cc:1909] T a4188bca839b4b238e05aaf78c7e58b0 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting consensus metadata
I20240502 20:46:17.521593 1727 ts_tablet_manager.cc:1888] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:17.521673 1727 log.cc:1196] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal/wals/a4188bca839b4b238e05aaf78c7e58b0
I20240502 20:46:17.521914 1727 ts_tablet_manager.cc:1909] T a4188bca839b4b238e05aaf78c7e58b0 P 315915efcf2b413ca7f9d874338e1b2d: Deleting consensus metadata
I20240502 20:46:17.522285 29028 catalog_manager.cc:4830] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): tablet a4188bca839b4b238e05aaf78c7e58b0 (table rebalancer_dynamic_table_new_name [id=d38f9ea0a0b6419abf445830750f9f47]) successfully deleted
I20240502 20:46:17.522809 29027 catalog_manager.cc:4830] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): tablet a4188bca839b4b238e05aaf78c7e58b0 (table rebalancer_dynamic_table_new_name [id=d38f9ea0a0b6419abf445830750f9f47]) successfully deleted
I20240502 20:46:17.526739 1728 tablet_replica.cc:330] T a4188bca839b4b238e05aaf78c7e58b0 P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:17.526902 1728 raft_consensus.cc:2229] T a4188bca839b4b238e05aaf78c7e58b0 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:17.527141 1728 raft_consensus.cc:2258] T a4188bca839b4b238e05aaf78c7e58b0 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:17.527719 1728 ts_tablet_manager.cc:1875] T a4188bca839b4b238e05aaf78c7e58b0 P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:17.529193 1720 tablet_bootstrap.cc:492] T 741da32f7c924490a5fa5933708adf30 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:17.529424 1720 tablet_bootstrap.cc:492] T 741da32f7c924490a5fa5933708adf30 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap complete.
I20240502 20:46:17.529569 1720 ts_tablet_manager.cc:1367] T 741da32f7c924490a5fa5933708adf30 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.026s user 0.019s sys 0.000s
I20240502 20:46:17.529791 1728 ts_tablet_manager.cc:1888] T a4188bca839b4b238e05aaf78c7e58b0 P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:17.529868 1728 log.cc:1196] T a4188bca839b4b238e05aaf78c7e58b0 P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/a4188bca839b4b238e05aaf78c7e58b0
I20240502 20:46:17.529975 1720 raft_consensus.cc:348] T 741da32f7c924490a5fa5933708adf30 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:17.530097 1720 raft_consensus.cc:729] T 741da32f7c924490a5fa5933708adf30 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:17.530164 1728 ts_tablet_manager.cc:1909] T a4188bca839b4b238e05aaf78c7e58b0 P e6b60803d0764d228c863e2da65e3691: Deleting consensus metadata
I20240502 20:46:17.530262 1720 consensus_queue.cc:260] T 741da32f7c924490a5fa5933708adf30 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:17.530989 1720 ts_tablet_manager.cc:1398] T 741da32f7c924490a5fa5933708adf30 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 20:46:17.531144 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet a4188bca839b4b238e05aaf78c7e58b0 (table rebalancer_dynamic_table_new_name [id=d38f9ea0a0b6419abf445830750f9f47]) successfully deleted
I20240502 20:46:17.532666 29335 tablet_copy_service.cc:342] P 09258ac095664fa99caf0707262cde13: Request end of tablet copy session 2a9c1af8cf814030bde56b67ca5c2fbe-741da32f7c924490a5fa5933708adf30 received from {username='slave'} at 127.22.45.69:52445
I20240502 20:46:17.532744 29335 tablet_copy_service.cc:433] P 09258ac095664fa99caf0707262cde13: ending tablet copy session 2a9c1af8cf814030bde56b67ca5c2fbe-741da32f7c924490a5fa5933708adf30 on tablet 741da32f7c924490a5fa5933708adf30 with peer 2a9c1af8cf814030bde56b67ca5c2fbe
I20240502 20:46:17.564996 1730 ts_tablet_manager.cc:902] T 4c4440d27d9e4070a8e425d26d8a0772 P e6b60803d0764d228c863e2da65e3691: Initiating tablet copy from peer dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:17.565454 1730 tablet_copy_client.cc:308] T 4c4440d27d9e4070a8e425d26d8a0772 P e6b60803d0764d228c863e2da65e3691: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.67:44095
I20240502 20:46:17.565936 29465 tablet_copy_service.cc:140] P dbcdc6ac635b44b185c3eba63f66e9d6: Received BeginTabletCopySession request for tablet 4c4440d27d9e4070a8e425d26d8a0772 from peer e6b60803d0764d228c863e2da65e3691 ({username='slave'} at 127.22.45.72:44455)
I20240502 20:46:17.566044 29465 tablet_copy_service.cc:161] P dbcdc6ac635b44b185c3eba63f66e9d6: Beginning new tablet copy session on tablet 4c4440d27d9e4070a8e425d26d8a0772 from peer e6b60803d0764d228c863e2da65e3691 at {username='slave'} at 127.22.45.72:44455: session id = e6b60803d0764d228c863e2da65e3691-4c4440d27d9e4070a8e425d26d8a0772
I20240502 20:46:17.566926 29465 tablet_copy_source_session.cc:205] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:17.567777 1730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c4440d27d9e4070a8e425d26d8a0772. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:17.569666 1730 tablet_copy_client.cc:783] T 4c4440d27d9e4070a8e425d26d8a0772 P e6b60803d0764d228c863e2da65e3691: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:17.569814 1730 tablet_copy_client.cc:647] T 4c4440d27d9e4070a8e425d26d8a0772 P e6b60803d0764d228c863e2da65e3691: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:17.570518 975 raft_consensus.cc:482] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:17.570614 975 raft_consensus.cc:504] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:17.570971 975 leader_election.cc:288] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:17.571274 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7db3d66689ef4fb2869e94448515ad12" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:17.571399 31888 raft_consensus.cc:2454] T 7db3d66689ef4fb2869e94448515ad12 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:17.571466 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7db3d66689ef4fb2869e94448515ad12" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:17.571383 1730 tablet_copy_client.cc:523] T 4c4440d27d9e4070a8e425d26d8a0772 P e6b60803d0764d228c863e2da65e3691: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:17.571602 31495 raft_consensus.cc:2454] T 7db3d66689ef4fb2869e94448515ad12 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:17.571682 31691 leader_election.cc:302] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e6b60803d0764d228c863e2da65e3691, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:17.571838 975 raft_consensus.cc:2781] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:17.571888 975 raft_consensus.cc:482] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:17.571923 975 raft_consensus.cc:3037] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:17.572875 975 raft_consensus.cc:504] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:17.573186 975 leader_election.cc:288] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Requested vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:17.573580 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7db3d66689ef4fb2869e94448515ad12" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:17.573608 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7db3d66689ef4fb2869e94448515ad12" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:17.573701 31888 raft_consensus.cc:3037] T 7db3d66689ef4fb2869e94448515ad12 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:17.573714 31495 raft_consensus.cc:3037] T 7db3d66689ef4fb2869e94448515ad12 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:17.573902 1730 tablet_bootstrap.cc:492] T 4c4440d27d9e4070a8e425d26d8a0772 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:17.574383 31888 raft_consensus.cc:2454] T 7db3d66689ef4fb2869e94448515ad12 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:17.574659 31691 leader_election.cc:302] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e6b60803d0764d228c863e2da65e3691, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:17.574779 31495 raft_consensus.cc:2454] T 7db3d66689ef4fb2869e94448515ad12 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:17.574826 975 raft_consensus.cc:2781] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:17.574899 975 raft_consensus.cc:686] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Becoming Leader. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Running, Role: LEADER
I20240502 20:46:17.575014 975 consensus_queue.cc:237] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:17.576916 29037 catalog_manager.cc:5484] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: term changed from 0 to 1, leader changed from <none> to e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71). New cstate: current_term: 1 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:17.597226 1730 tablet_bootstrap.cc:492] T 4c4440d27d9e4070a8e425d26d8a0772 P e6b60803d0764d228c863e2da65e3691: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:17.597599 1730 tablet_bootstrap.cc:492] T 4c4440d27d9e4070a8e425d26d8a0772 P e6b60803d0764d228c863e2da65e3691: Bootstrap complete.
I20240502 20:46:17.597805 1730 ts_tablet_manager.cc:1367] T 4c4440d27d9e4070a8e425d26d8a0772 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.024s user 0.021s sys 0.000s
I20240502 20:46:17.598290 1730 raft_consensus.cc:348] T 4c4440d27d9e4070a8e425d26d8a0772 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:17.598417 1730 raft_consensus.cc:729] T 4c4440d27d9e4070a8e425d26d8a0772 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:17.598624 1730 consensus_queue.cc:260] T 4c4440d27d9e4070a8e425d26d8a0772 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:17.599237 1730 ts_tablet_manager.cc:1398] T 4c4440d27d9e4070a8e425d26d8a0772 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 20:46:17.599807 29293 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:17.599910 29465 tablet_copy_service.cc:342] P dbcdc6ac635b44b185c3eba63f66e9d6: Request end of tablet copy session e6b60803d0764d228c863e2da65e3691-4c4440d27d9e4070a8e425d26d8a0772 received from {username='slave'} at 127.22.45.72:44455
I20240502 20:46:17.599992 29465 tablet_copy_service.cc:433] P dbcdc6ac635b44b185c3eba63f66e9d6: ending tablet copy session e6b60803d0764d228c863e2da65e3691-4c4440d27d9e4070a8e425d26d8a0772 on tablet 4c4440d27d9e4070a8e425d26d8a0772 with peer e6b60803d0764d228c863e2da65e3691
I20240502 20:46:17.603052 31475 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:17.616005 31602 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:17.616039 29162 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:17.616858 29037 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:17.619208 29426 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:17.619735 29557 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:17.631253 29557 tablet_service.cc:1467] Processing CreateTablet for tablet 773925f81a4147a6a0712ec84a2b3ff0 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=808114c9773543fd8ad8c8c967fcb071]), partition=
I20240502 20:46:17.631373 1694 ts_tablet_manager.cc:902] T 11ee3029cd744581ba9c5c213ca8fd2b P 09258ac095664fa99caf0707262cde13: Initiating tablet copy from peer e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:17.631464 31735 tablet_service.cc:1467] Processing CreateTablet for tablet 773925f81a4147a6a0712ec84a2b3ff0 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=808114c9773543fd8ad8c8c967fcb071]), partition=
I20240502 20:46:17.631588 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 773925f81a4147a6a0712ec84a2b3ff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:17.631693 31735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 773925f81a4147a6a0712ec84a2b3ff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:17.631839 1694 tablet_copy_client.cc:308] T 11ee3029cd744581ba9c5c213ca8fd2b P 09258ac095664fa99caf0707262cde13: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.72:35207
I20240502 20:46:17.632230 31475 tablet_service.cc:1467] Processing CreateTablet for tablet 773925f81a4147a6a0712ec84a2b3ff0 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=808114c9773543fd8ad8c8c967fcb071]), partition=
I20240502 20:46:17.632504 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 773925f81a4147a6a0712ec84a2b3ff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:17.633882 1670 tablet_bootstrap.cc:492] T 773925f81a4147a6a0712ec84a2b3ff0 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:17.634588 1154 tablet_bootstrap.cc:492] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:17.634754 1670 tablet_bootstrap.cc:654] T 773925f81a4147a6a0712ec84a2b3ff0 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:17.635108 31908 tablet_copy_service.cc:140] P e6b60803d0764d228c863e2da65e3691: Received BeginTabletCopySession request for tablet 11ee3029cd744581ba9c5c213ca8fd2b from peer 09258ac095664fa99caf0707262cde13 ({username='slave'} at 127.22.45.66:55365)
I20240502 20:46:17.635208 31908 tablet_copy_service.cc:161] P e6b60803d0764d228c863e2da65e3691: Beginning new tablet copy session on tablet 11ee3029cd744581ba9c5c213ca8fd2b from peer 09258ac095664fa99caf0707262cde13 at {username='slave'} at 127.22.45.66:55365: session id = 09258ac095664fa99caf0707262cde13-11ee3029cd744581ba9c5c213ca8fd2b
I20240502 20:46:17.635432 1154 tablet_bootstrap.cc:654] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:17.636122 31908 tablet_copy_source_session.cc:205] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:17.636950 1694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11ee3029cd744581ba9c5c213ca8fd2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:17.639117 1694 tablet_copy_client.cc:783] T 11ee3029cd744581ba9c5c213ca8fd2b P 09258ac095664fa99caf0707262cde13: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:17.639240 1694 tablet_copy_client.cc:647] T 11ee3029cd744581ba9c5c213ca8fd2b P 09258ac095664fa99caf0707262cde13: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:17.644170 1694 tablet_copy_client.cc:523] T 11ee3029cd744581ba9c5c213ca8fd2b P 09258ac095664fa99caf0707262cde13: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:17.644304 1752 tablet_bootstrap.cc:492] T 773925f81a4147a6a0712ec84a2b3ff0 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:17.645439 1752 tablet_bootstrap.cc:654] T 773925f81a4147a6a0712ec84a2b3ff0 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:17.645690 1694 tablet_bootstrap.cc:492] T 11ee3029cd744581ba9c5c213ca8fd2b P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:17.654845 31867 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:17.656278 1694 tablet_bootstrap.cc:492] T 11ee3029cd744581ba9c5c213ca8fd2b P 09258ac095664fa99caf0707262cde13: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:17.656524 1694 tablet_bootstrap.cc:492] T 11ee3029cd744581ba9c5c213ca8fd2b P 09258ac095664fa99caf0707262cde13: Bootstrap complete.
I20240502 20:46:17.656690 1694 ts_tablet_manager.cc:1367] T 11ee3029cd744581ba9c5c213ca8fd2b P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
I20240502 20:46:17.657164 1694 raft_consensus.cc:348] T 11ee3029cd744581ba9c5c213ca8fd2b P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:17.657287 1694 raft_consensus.cc:729] T 11ee3029cd744581ba9c5c213ca8fd2b P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:17.657423 1694 consensus_queue.cc:260] T 11ee3029cd744581ba9c5c213ca8fd2b P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:17.658253 1694 ts_tablet_manager.cc:1398] T 11ee3029cd744581ba9c5c213ca8fd2b P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240502 20:46:17.658754 31908 tablet_copy_service.cc:342] P e6b60803d0764d228c863e2da65e3691: Request end of tablet copy session 09258ac095664fa99caf0707262cde13-11ee3029cd744581ba9c5c213ca8fd2b received from {username='slave'} at 127.22.45.66:55365
I20240502 20:46:17.658830 31908 tablet_copy_service.cc:433] P e6b60803d0764d228c863e2da65e3691: ending tablet copy session 09258ac095664fa99caf0707262cde13-11ee3029cd744581ba9c5c213ca8fd2b on tablet 11ee3029cd744581ba9c5c213ca8fd2b with peer 09258ac095664fa99caf0707262cde13
I20240502 20:46:17.659245 1154 tablet_bootstrap.cc:492] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:17.659335 1154 ts_tablet_manager.cc:1367] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.025s user 0.000s sys 0.002s
I20240502 20:46:17.659512 1670 tablet_bootstrap.cc:492] T 773925f81a4147a6a0712ec84a2b3ff0 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:17.659608 1670 ts_tablet_manager.cc:1367] T 773925f81a4147a6a0712ec84a2b3ff0 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.026s user 0.001s sys 0.001s
I20240502 20:46:17.659795 1154 raft_consensus.cc:348] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:17.659897 1154 raft_consensus.cc:374] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:17.659927 1154 raft_consensus.cc:729] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:17.660032 1670 raft_consensus.cc:348] T 773925f81a4147a6a0712ec84a2b3ff0 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:17.660048 1154 consensus_queue.cc:260] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:17.660135 1670 raft_consensus.cc:374] T 773925f81a4147a6a0712ec84a2b3ff0 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:17.660166 1670 raft_consensus.cc:729] T 773925f81a4147a6a0712ec84a2b3ff0 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:17.660274 1670 consensus_queue.cc:260] T 773925f81a4147a6a0712ec84a2b3ff0 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:17.660351 1154 ts_tablet_manager.cc:1398] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:17.660531 1670 ts_tablet_manager.cc:1398] T 773925f81a4147a6a0712ec84a2b3ff0 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:17.663249 31735 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:17.664297 1752 tablet_bootstrap.cc:492] T 773925f81a4147a6a0712ec84a2b3ff0 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:17.664456 1752 ts_tablet_manager.cc:1367] T 773925f81a4147a6a0712ec84a2b3ff0 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20240502 20:46:17.665004 1752 raft_consensus.cc:348] T 773925f81a4147a6a0712ec84a2b3ff0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:17.665100 1752 raft_consensus.cc:374] T 773925f81a4147a6a0712ec84a2b3ff0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:17.665127 1752 raft_consensus.cc:729] T 773925f81a4147a6a0712ec84a2b3ff0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:17.665328 1752 consensus_queue.cc:260] T 773925f81a4147a6a0712ec84a2b3ff0 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:17.665656 1752 ts_tablet_manager.cc:1398] T 773925f81a4147a6a0712ec84a2b3ff0 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:17.710711 29182 consensus_queue.cc:237] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:17.712185 31888 raft_consensus.cc:1261] T a0e1f5ce5c544045b556cb31e744b169 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:17.712584 29574 raft_consensus.cc:1261] T a0e1f5ce5c544045b556cb31e744b169 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:17.712633 1773 consensus_queue.cc:1035] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:17.713003 1093 consensus_queue.cc:1035] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:17.714499 1773 raft_consensus.cc:2932] T a0e1f5ce5c544045b556cb31e744b169 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:17.715301 31888 raft_consensus.cc:2932] T a0e1f5ce5c544045b556cb31e744b169 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:17.717903 29041 catalog_manager.cc:5484] T a0e1f5ce5c544045b556cb31e744b169 P e6b60803d0764d228c863e2da65e3691 reported cstate change: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:17.718533 29574 raft_consensus.cc:2932] T a0e1f5ce5c544045b556cb31e744b169 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:17.720855 29426 tablet_service.cc:1514] Processing DeleteTablet for tablet a0e1f5ce5c544045b556cb31e744b169 with delete_type TABLET_DATA_TOMBSTONED (TS dbcdc6ac635b44b185c3eba63f66e9d6 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:17.721414 1779 tablet_replica.cc:330] T a0e1f5ce5c544045b556cb31e744b169 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:17.721691 1779 raft_consensus.cc:2229] T a0e1f5ce5c544045b556cb31e744b169 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:17.721994 1779 raft_consensus.cc:2258] T a0e1f5ce5c544045b556cb31e744b169 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:17.722806 1779 ts_tablet_manager.cc:1875] T a0e1f5ce5c544045b556cb31e744b169 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:17.725041 1779 ts_tablet_manager.cc:1888] T a0e1f5ce5c544045b556cb31e744b169 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 20:46:17.725216 1779 log.cc:1196] T a0e1f5ce5c544045b556cb31e744b169 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/a0e1f5ce5c544045b556cb31e744b169
I20240502 20:46:17.725725 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet a0e1f5ce5c544045b556cb31e744b169 (table rebalance_test_table_11 [id=cc807ad977d146dc84be7ee2897657fc]) successfully deleted
I20240502 20:46:17.813333 32108 raft_consensus.cc:482] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:17.813465 32108 raft_consensus.cc:504] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:17.813944 32108 leader_election.cc:288] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:17.814396 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "773925f81a4147a6a0712ec84a2b3ff0" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:17.814585 31757 raft_consensus.cc:2454] T 773925f81a4147a6a0712ec84a2b3ff0 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:17.814946 31428 leader_election.cc:302] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:17.815097 32108 raft_consensus.cc:2781] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:17.815145 32108 raft_consensus.cc:482] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:17.815193 32108 raft_consensus.cc:3037] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:17.817744 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "773925f81a4147a6a0712ec84a2b3ff0" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:17.817878 29574 raft_consensus.cc:2454] T 773925f81a4147a6a0712ec84a2b3ff0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:17.818917 32108 raft_consensus.cc:504] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:17.819355 32108 leader_election.cc:288] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:17.819676 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "773925f81a4147a6a0712ec84a2b3ff0" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:17.819783 29574 raft_consensus.cc:3037] T 773925f81a4147a6a0712ec84a2b3ff0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:17.819731 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "773925f81a4147a6a0712ec84a2b3ff0" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:17.819870 31757 raft_consensus.cc:3037] T 773925f81a4147a6a0712ec84a2b3ff0 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:17.820622 31757 raft_consensus.cc:2454] T 773925f81a4147a6a0712ec84a2b3ff0 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:17.820683 29574 raft_consensus.cc:2454] T 773925f81a4147a6a0712ec84a2b3ff0 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:17.820956 31429 leader_election.cc:302] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:17.821162 32108 raft_consensus.cc:2781] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:17.821238 32108 raft_consensus.cc:686] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Becoming Leader. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Running, Role: LEADER
I20240502 20:46:17.821374 32108 consensus_queue.cc:237] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:17.823249 29035 catalog_manager.cc:5484] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69). New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:17.840257 29293 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:17.843935 31475 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:17.847113 31735 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:17.853139 29557 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:17.864966 29162 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:17.865633 29426 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:17.868121 31867 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:17.895853 31602 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:17.948616 822 consensus_queue.cc:1035] T a58b3a6ce8c042c8bd371d52e1f2a270 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:17.956238 1844 consensus_queue.cc:1035] T a58b3a6ce8c042c8bd371d52e1f2a270 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:17.965586 29446 consensus_queue.cc:237] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:17.966966 29314 raft_consensus.cc:1261] T 9d2cd90615dc4d1894ffe5e80914bdc0 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:17.967366 31757 raft_consensus.cc:1261] T 9d2cd90615dc4d1894ffe5e80914bdc0 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:17.967401 1503 consensus_queue.cc:1035] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:17.967751 1503 consensus_queue.cc:1035] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:17.968073 29042 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:17.969317 1503 raft_consensus.cc:2932] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:17.969834 31757 raft_consensus.cc:2932] T 9d2cd90615dc4d1894ffe5e80914bdc0 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:17.969834 29314 raft_consensus.cc:2932] T 9d2cd90615dc4d1894ffe5e80914bdc0 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:17.972445 29035 catalog_manager.cc:5484] T 9d2cd90615dc4d1894ffe5e80914bdc0 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index 4 to 5, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) evicted. New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:17.975581 29574 raft_consensus.cc:1261] T 773925f81a4147a6a0712ec84a2b3ff0 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:17.976305 32108 consensus_queue.cc:1035] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:17.976372 29557 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d2cd90615dc4d1894ffe5e80914bdc0 with delete_type TABLET_DATA_TOMBSTONED (TS db17f26a73f3447b92aa9a947e3a7057 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:17.976692 1859 tablet_replica.cc:330] T 9d2cd90615dc4d1894ffe5e80914bdc0 P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:17.976873 1859 raft_consensus.cc:2229] T 9d2cd90615dc4d1894ffe5e80914bdc0 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:17.977061 1859 raft_consensus.cc:2258] T 9d2cd90615dc4d1894ffe5e80914bdc0 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:17.977614 1859 ts_tablet_manager.cc:1875] T 9d2cd90615dc4d1894ffe5e80914bdc0 P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:17.978919 29034 catalog_manager.cc:5576] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897) does not have the latest schema for tablet 773925f81a4147a6a0712ec84a2b3ff0 (table rebalancer_dynamic_table [id=808114c9773543fd8ad8c8c967fcb071]). Expected version 1 got 0
I20240502 20:46:17.979846 1863 tablet.cc:1722] T 773925f81a4147a6a0712ec84a2b3ff0 P db17f26a73f3447b92aa9a947e3a7057: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:17.980166 31757 raft_consensus.cc:1261] T 773925f81a4147a6a0712ec84a2b3ff0 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:17.980263 1863 tablet.cc:1620] T 773925f81a4147a6a0712ec84a2b3ff0 P db17f26a73f3447b92aa9a947e3a7057: MemRowSet was empty: no flush needed.
I20240502 20:46:17.980664 32108 consensus_queue.cc:1035] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:17.980727 1859 ts_tablet_manager.cc:1888] T 9d2cd90615dc4d1894ffe5e80914bdc0 P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 20:46:17.980783 1859 log.cc:1196] T 9d2cd90615dc4d1894ffe5e80914bdc0 P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/9d2cd90615dc4d1894ffe5e80914bdc0
I20240502 20:46:17.980883 1862 tablet.cc:1722] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:17.981236 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet 9d2cd90615dc4d1894ffe5e80914bdc0 (table rebalance_test_table_5 [id=fd2f36435a614aa98d7f33e9646f33a3]) successfully deleted
I20240502 20:46:17.982136 1862 tablet.cc:1620] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:17.983405 1870 tablet.cc:1722] T 773925f81a4147a6a0712ec84a2b3ff0 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:17.983678 1870 tablet.cc:1620] T 773925f81a4147a6a0712ec84a2b3ff0 P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:17.983667 29034 catalog_manager.cc:5576] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787) does not have the latest schema for tablet 773925f81a4147a6a0712ec84a2b3ff0 (table rebalancer_dynamic_table [id=808114c9773543fd8ad8c8c967fcb071]). Expected version 1 got 0
I20240502 20:46:17.987495 29034 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=808114c9773543fd8ad8c8c967fcb071] alter complete (version 1)
I20240502 20:46:17.989964 29314 raft_consensus.cc:1203] T 0f57649f122e45f9b8ef82ec975e5359 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 20:46:18.003755 32108 consensus_queue.cc:1035] T 2e4a0e5ef89d447b96b998f085e3495c P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:18.008733 32108 consensus_queue.cc:1035] T 2e4a0e5ef89d447b96b998f085e3495c P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:18.017094 975 consensus_queue.cc:1035] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:18.020754 692 consensus_queue.cc:1035] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:18.043268 31495 raft_consensus.cc:1203] T 741da32f7c924490a5fa5933708adf30 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:18.063547 29314 raft_consensus.cc:1203] T 11ee3029cd744581ba9c5c213ca8fd2b P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 20:46:18.090615 29038 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:18.099099 31602 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:18.103643 29037 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_16"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:18.104773 1862 tablet.cc:1722] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:18.105374 1862 tablet.cc:1620] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:18.109732 29162 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:18.109997 1870 tablet.cc:1722] T 773925f81a4147a6a0712ec84a2b3ff0 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:18.110332 1870 tablet.cc:1620] T 773925f81a4147a6a0712ec84a2b3ff0 P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:18.111253 1863 tablet.cc:1722] T 773925f81a4147a6a0712ec84a2b3ff0 P db17f26a73f3447b92aa9a947e3a7057: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:18.111541 1863 tablet.cc:1620] T 773925f81a4147a6a0712ec84a2b3ff0 P db17f26a73f3447b92aa9a947e3a7057: MemRowSet was empty: no flush needed.
I20240502 20:46:18.112854 31475 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:18.114521 29557 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:18.119288 29426 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:18.122886 29042 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=808114c9773543fd8ad8c8c967fcb071] alter complete (version 2)
I20240502 20:46:18.127727 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 80e02f0cb68f44969be535458d283f3e (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=258f1a0190814114b9d8e7f4437b10da]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:18.128016 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80e02f0cb68f44969be535458d283f3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:18.128274 31474 tablet_service.cc:1467] Processing CreateTablet for tablet 80e02f0cb68f44969be535458d283f3e (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=258f1a0190814114b9d8e7f4437b10da]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:18.128535 31474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80e02f0cb68f44969be535458d283f3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:18.130424 29162 tablet_service.cc:1467] Processing CreateTablet for tablet e90d62851df5410481db24b0ab26a274 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=258f1a0190814114b9d8e7f4437b10da]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:18.130584 1154 tablet_bootstrap.cc:492] T 80e02f0cb68f44969be535458d283f3e P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:18.130717 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e90d62851df5410481db24b0ab26a274. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:18.131011 31602 tablet_service.cc:1467] Processing CreateTablet for tablet 46c0a5b0870c4a099c7c17ebc0a5d34e (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=258f1a0190814114b9d8e7f4437b10da]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:18.131292 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46c0a5b0870c4a099c7c17ebc0a5d34e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:18.131440 1154 tablet_bootstrap.cc:654] T 80e02f0cb68f44969be535458d283f3e P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:18.132344 1943 tablet_bootstrap.cc:492] T 80e02f0cb68f44969be535458d283f3e P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:18.133369 31867 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:18.133708 1945 tablet_bootstrap.cc:492] T e90d62851df5410481db24b0ab26a274 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:18.134001 29557 tablet_service.cc:1467] Processing CreateTablet for tablet e90d62851df5410481db24b0ab26a274 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=258f1a0190814114b9d8e7f4437b10da]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:18.134266 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e90d62851df5410481db24b0ab26a274. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:18.135025 29426 tablet_service.cc:1467] Processing CreateTablet for tablet 46c0a5b0870c4a099c7c17ebc0a5d34e (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=258f1a0190814114b9d8e7f4437b10da]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:18.135299 29426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46c0a5b0870c4a099c7c17ebc0a5d34e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:18.135321 31475 tablet_service.cc:1467] Processing CreateTablet for tablet e90d62851df5410481db24b0ab26a274 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=258f1a0190814114b9d8e7f4437b10da]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:18.135387 31735 tablet_service.cc:1467] Processing CreateTablet for tablet 46c0a5b0870c4a099c7c17ebc0a5d34e (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=258f1a0190814114b9d8e7f4437b10da]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:18.135545 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e90d62851df5410481db24b0ab26a274. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:18.135679 31735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46c0a5b0870c4a099c7c17ebc0a5d34e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:18.136246 1943 tablet_bootstrap.cc:654] T 80e02f0cb68f44969be535458d283f3e P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:18.136915 1946 tablet_bootstrap.cc:492] T 46c0a5b0870c4a099c7c17ebc0a5d34e P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:18.137815 1670 tablet_bootstrap.cc:492] T 46c0a5b0870c4a099c7c17ebc0a5d34e P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:18.137817 1946 tablet_bootstrap.cc:654] T 46c0a5b0870c4a099c7c17ebc0a5d34e P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:18.137811 31887 raft_consensus.cc:1203] T 4c4440d27d9e4070a8e425d26d8a0772 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:18.138453 1945 tablet_bootstrap.cc:654] T e90d62851df5410481db24b0ab26a274 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:18.138695 1670 tablet_bootstrap.cc:654] T 46c0a5b0870c4a099c7c17ebc0a5d34e P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:18.135025 29424 tablet_service.cc:1467] Processing CreateTablet for tablet 80e02f0cb68f44969be535458d283f3e (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=258f1a0190814114b9d8e7f4437b10da]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:18.139011 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80e02f0cb68f44969be535458d283f3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:18.139994 1670 tablet_bootstrap.cc:492] T 46c0a5b0870c4a099c7c17ebc0a5d34e P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:18.140079 1670 ts_tablet_manager.cc:1367] T 46c0a5b0870c4a099c7c17ebc0a5d34e P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:18.140506 1670 raft_consensus.cc:348] T 46c0a5b0870c4a099c7c17ebc0a5d34e P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:18.140619 1670 raft_consensus.cc:374] T 46c0a5b0870c4a099c7c17ebc0a5d34e P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:18.140650 1670 raft_consensus.cc:729] T 46c0a5b0870c4a099c7c17ebc0a5d34e P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:18.140767 1670 consensus_queue.cc:260] T 46c0a5b0870c4a099c7c17ebc0a5d34e P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:18.141041 1670 ts_tablet_manager.cc:1398] T 46c0a5b0870c4a099c7c17ebc0a5d34e P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:18.141587 1945 tablet_bootstrap.cc:492] T e90d62851df5410481db24b0ab26a274 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:18.141685 1945 ts_tablet_manager.cc:1367] T e90d62851df5410481db24b0ab26a274 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 20:46:18.141942 1752 tablet_bootstrap.cc:492] T e90d62851df5410481db24b0ab26a274 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:18.142097 1945 raft_consensus.cc:348] T e90d62851df5410481db24b0ab26a274 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:18.142213 1945 raft_consensus.cc:374] T e90d62851df5410481db24b0ab26a274 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:18.142247 1945 raft_consensus.cc:729] T e90d62851df5410481db24b0ab26a274 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:18.142369 1945 consensus_queue.cc:260] T e90d62851df5410481db24b0ab26a274 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:18.142663 1945 ts_tablet_manager.cc:1398] T e90d62851df5410481db24b0ab26a274 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:18.142853 1752 tablet_bootstrap.cc:654] T e90d62851df5410481db24b0ab26a274 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:18.144096 1752 tablet_bootstrap.cc:492] T e90d62851df5410481db24b0ab26a274 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:18.144186 1752 ts_tablet_manager.cc:1367] T e90d62851df5410481db24b0ab26a274 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:18.144623 1752 raft_consensus.cc:348] T e90d62851df5410481db24b0ab26a274 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:18.144714 1752 raft_consensus.cc:374] T e90d62851df5410481db24b0ab26a274 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:18.144740 1752 raft_consensus.cc:729] T e90d62851df5410481db24b0ab26a274 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:18.144869 1752 consensus_queue.cc:260] T e90d62851df5410481db24b0ab26a274 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:18.145128 1752 ts_tablet_manager.cc:1398] T e90d62851df5410481db24b0ab26a274 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:18.153271 1384 raft_consensus.cc:482] T e90d62851df5410481db24b0ab26a274 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:18.153374 1384 raft_consensus.cc:504] T e90d62851df5410481db24b0ab26a274 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:18.153800 1384 leader_election.cc:288] T e90d62851df5410481db24b0ab26a274 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:18.154209 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e90d62851df5410481db24b0ab26a274" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
W20240502 20:46:18.154690 29512 leader_election.cc:341] T e90d62851df5410481db24b0ab26a274 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): Illegal state: must be running to vote when last-logged opid is not known
I20240502 20:46:18.155771 31735 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:18.159207 1943 tablet_bootstrap.cc:492] T 80e02f0cb68f44969be535458d283f3e P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:18.159337 1943 ts_tablet_manager.cc:1367] T 80e02f0cb68f44969be535458d283f3e P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.000s
I20240502 20:46:18.159858 1943 raft_consensus.cc:348] T 80e02f0cb68f44969be535458d283f3e P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:18.159976 1943 raft_consensus.cc:374] T 80e02f0cb68f44969be535458d283f3e P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:18.160008 1943 raft_consensus.cc:729] T 80e02f0cb68f44969be535458d283f3e P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:18.160144 1943 consensus_queue.cc:260] T 80e02f0cb68f44969be535458d283f3e P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:18.160450 1943 ts_tablet_manager.cc:1398] T 80e02f0cb68f44969be535458d283f3e P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:18.162633 1948 tablet_bootstrap.cc:492] T 46c0a5b0870c4a099c7c17ebc0a5d34e P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:18.162698 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e90d62851df5410481db24b0ab26a274" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:18.162832 29182 raft_consensus.cc:2454] T e90d62851df5410481db24b0ab26a274 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:18.163007 1154 tablet_bootstrap.cc:492] T 80e02f0cb68f44969be535458d283f3e P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:18.163093 1154 ts_tablet_manager.cc:1367] T 80e02f0cb68f44969be535458d283f3e P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.033s user 0.001s sys 0.000s
I20240502 20:46:18.163580 1948 tablet_bootstrap.cc:654] T 46c0a5b0870c4a099c7c17ebc0a5d34e P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:18.163537 1154 raft_consensus.cc:348] T 80e02f0cb68f44969be535458d283f3e P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:18.163642 1154 raft_consensus.cc:374] T 80e02f0cb68f44969be535458d283f3e P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:18.163681 1154 raft_consensus.cc:729] T 80e02f0cb68f44969be535458d283f3e P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:18.163802 1154 consensus_queue.cc:260] T 80e02f0cb68f44969be535458d283f3e P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:18.164067 1154 ts_tablet_manager.cc:1398] T 80e02f0cb68f44969be535458d283f3e P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:18.164238 1154 tablet_bootstrap.cc:492] T e90d62851df5410481db24b0ab26a274 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:18.165110 1154 tablet_bootstrap.cc:654] T e90d62851df5410481db24b0ab26a274 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:18.166980 1154 tablet_bootstrap.cc:492] T e90d62851df5410481db24b0ab26a274 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:18.167734 29513 leader_election.cc:302] T e90d62851df5410481db24b0ab26a274 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057; no voters: 2a9c1af8cf814030bde56b67ca5c2fbe
I20240502 20:46:18.167866 1154 ts_tablet_manager.cc:1367] T e90d62851df5410481db24b0ab26a274 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:18.167940 1384 raft_consensus.cc:2781] T e90d62851df5410481db24b0ab26a274 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:18.167999 1384 raft_consensus.cc:482] T e90d62851df5410481db24b0ab26a274 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:18.168037 1384 raft_consensus.cc:3037] T e90d62851df5410481db24b0ab26a274 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:18.168406 1154 raft_consensus.cc:348] T e90d62851df5410481db24b0ab26a274 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:18.168494 1154 raft_consensus.cc:374] T e90d62851df5410481db24b0ab26a274 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:18.168519 1154 raft_consensus.cc:729] T e90d62851df5410481db24b0ab26a274 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:18.168649 1154 consensus_queue.cc:260] T e90d62851df5410481db24b0ab26a274 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:18.168911 1154 ts_tablet_manager.cc:1398] T e90d62851df5410481db24b0ab26a274 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:18.169102 1384 raft_consensus.cc:504] T e90d62851df5410481db24b0ab26a274 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:18.169412 1384 leader_election.cc:288] T e90d62851df5410481db24b0ab26a274 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Requested vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:18.169795 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e90d62851df5410481db24b0ab26a274" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:18.169893 31495 raft_consensus.cc:3037] T e90d62851df5410481db24b0ab26a274 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:18.170069 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e90d62851df5410481db24b0ab26a274" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:18.170181 29182 raft_consensus.cc:3037] T e90d62851df5410481db24b0ab26a274 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:18.170882 31495 raft_consensus.cc:2454] T e90d62851df5410481db24b0ab26a274 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:18.171177 29182 raft_consensus.cc:2454] T e90d62851df5410481db24b0ab26a274 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:18.171823 29512 leader_election.cc:302] T e90d62851df5410481db24b0ab26a274 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:18.171986 1384 raft_consensus.cc:2781] T e90d62851df5410481db24b0ab26a274 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:18.172048 1384 raft_consensus.cc:686] T e90d62851df5410481db24b0ab26a274 P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Becoming Leader. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Running, Role: LEADER
I20240502 20:46:18.172174 1384 consensus_queue.cc:237] T e90d62851df5410481db24b0ab26a274 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:18.175149 29038 catalog_manager.cc:5484] T e90d62851df5410481db24b0ab26a274 P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: term changed from 0 to 1, leader changed from <none> to db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:18.178820 29293 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:18.197234 1946 tablet_bootstrap.cc:492] T 46c0a5b0870c4a099c7c17ebc0a5d34e P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:18.197355 1946 ts_tablet_manager.cc:1367] T 46c0a5b0870c4a099c7c17ebc0a5d34e P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.060s user 0.002s sys 0.000s
I20240502 20:46:18.197863 1946 raft_consensus.cc:348] T 46c0a5b0870c4a099c7c17ebc0a5d34e P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:18.197983 1946 raft_consensus.cc:374] T 46c0a5b0870c4a099c7c17ebc0a5d34e P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:18.198010 1946 raft_consensus.cc:729] T 46c0a5b0870c4a099c7c17ebc0a5d34e P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:18.198699 1948 tablet_bootstrap.cc:492] T 46c0a5b0870c4a099c7c17ebc0a5d34e P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:18.198799 1948 ts_tablet_manager.cc:1367] T 46c0a5b0870c4a099c7c17ebc0a5d34e P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.036s user 0.002s sys 0.000s
I20240502 20:46:18.199219 1948 raft_consensus.cc:348] T 46c0a5b0870c4a099c7c17ebc0a5d34e P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:18.199326 1948 raft_consensus.cc:374] T 46c0a5b0870c4a099c7c17ebc0a5d34e P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:18.199354 1948 raft_consensus.cc:729] T 46c0a5b0870c4a099c7c17ebc0a5d34e P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:18.199463 1948 consensus_queue.cc:260] T 46c0a5b0870c4a099c7c17ebc0a5d34e P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:18.199713 1948 ts_tablet_manager.cc:1398] T 46c0a5b0870c4a099c7c17ebc0a5d34e P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:18.199863 1948 tablet_bootstrap.cc:492] T 80e02f0cb68f44969be535458d283f3e P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:18.200177 1946 consensus_queue.cc:260] T 46c0a5b0870c4a099c7c17ebc0a5d34e P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:18.200790 1948 tablet_bootstrap.cc:654] T 80e02f0cb68f44969be535458d283f3e P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:18.201287 1946 ts_tablet_manager.cc:1398] T 46c0a5b0870c4a099c7c17ebc0a5d34e P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:18.207129 1948 tablet_bootstrap.cc:492] T 80e02f0cb68f44969be535458d283f3e P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:18.207243 1948 ts_tablet_manager.cc:1367] T 80e02f0cb68f44969be535458d283f3e P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20240502 20:46:18.207698 1948 raft_consensus.cc:348] T 80e02f0cb68f44969be535458d283f3e P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:18.207798 1948 raft_consensus.cc:374] T 80e02f0cb68f44969be535458d283f3e P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:18.207830 1948 raft_consensus.cc:729] T 80e02f0cb68f44969be535458d283f3e P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:18.207945 1948 consensus_queue.cc:260] T 80e02f0cb68f44969be535458d283f3e P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:18.208226 1948 ts_tablet_manager.cc:1398] T 80e02f0cb68f44969be535458d283f3e P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:18.226447 29038 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:18.232055 1862 tablet.cc:1722] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:18.232887 1863 tablet.cc:1722] T 773925f81a4147a6a0712ec84a2b3ff0 P db17f26a73f3447b92aa9a947e3a7057: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:18.233258 1870 tablet.cc:1722] T 773925f81a4147a6a0712ec84a2b3ff0 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:18.237943 29038 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=808114c9773543fd8ad8c8c967fcb071] alter complete (version 3)
I20240502 20:46:18.239868 29446 consensus_queue.cc:237] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:18.241127 29574 raft_consensus.cc:1261] T 4c4440d27d9e4070a8e425d26d8a0772 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:18.241477 31887 raft_consensus.cc:1261] T 4c4440d27d9e4070a8e425d26d8a0772 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:18.241487 1321 consensus_queue.cc:1035] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:18.241950 1321 consensus_queue.cc:1035] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:18.243149 1857 raft_consensus.cc:2932] T 4c4440d27d9e4070a8e425d26d8a0772 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:18.243577 29574 raft_consensus.cc:2932] T 4c4440d27d9e4070a8e425d26d8a0772 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:18.244089 31887 raft_consensus.cc:2932] T 4c4440d27d9e4070a8e425d26d8a0772 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:18.246279 29041 catalog_manager.cc:5484] T 4c4440d27d9e4070a8e425d26d8a0772 P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: config changed from index 4 to 5, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) evicted. New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:18.249256 29162 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c4440d27d9e4070a8e425d26d8a0772 with delete_type TABLET_DATA_TOMBSTONED (TS 7796ea3fba294d17a34386873969431d not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:18.249639 1975 tablet_replica.cc:330] T 4c4440d27d9e4070a8e425d26d8a0772 P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:18.249799 1975 raft_consensus.cc:2229] T 4c4440d27d9e4070a8e425d26d8a0772 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:18.249960 1975 raft_consensus.cc:2258] T 4c4440d27d9e4070a8e425d26d8a0772 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:18.250524 1975 ts_tablet_manager.cc:1875] T 4c4440d27d9e4070a8e425d26d8a0772 P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:18.253037 1975 ts_tablet_manager.cc:1888] T 4c4440d27d9e4070a8e425d26d8a0772 P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 20:46:18.253121 1975 log.cc:1196] T 4c4440d27d9e4070a8e425d26d8a0772 P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/4c4440d27d9e4070a8e425d26d8a0772
I20240502 20:46:18.253857 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 4c4440d27d9e4070a8e425d26d8a0772 (table rebalance_test_table_17 [id=b1432bc7242c45e38bf75a9c91c6646c]) successfully deleted
I20240502 20:46:18.327128 1857 raft_consensus.cc:482] T 80e02f0cb68f44969be535458d283f3e P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:18.327270 1857 raft_consensus.cc:504] T 80e02f0cb68f44969be535458d283f3e P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:18.327795 1857 leader_election.cc:288] T 80e02f0cb68f44969be535458d283f3e P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:18.328310 31887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e02f0cb68f44969be535458d283f3e" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:18.328460 31887 raft_consensus.cc:2454] T 80e02f0cb68f44969be535458d283f3e P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:18.329075 29380 leader_election.cc:302] T 80e02f0cb68f44969be535458d283f3e P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dbcdc6ac635b44b185c3eba63f66e9d6, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:18.329416 1857 raft_consensus.cc:2781] T 80e02f0cb68f44969be535458d283f3e P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:18.329473 1857 raft_consensus.cc:482] T 80e02f0cb68f44969be535458d283f3e P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:18.329509 1857 raft_consensus.cc:3037] T 80e02f0cb68f44969be535458d283f3e P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:18.330689 1857 raft_consensus.cc:504] T 80e02f0cb68f44969be535458d283f3e P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:18.331060 1857 leader_election.cc:288] T 80e02f0cb68f44969be535458d283f3e P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:18.331593 31887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e02f0cb68f44969be535458d283f3e" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:18.331714 31887 raft_consensus.cc:3037] T 80e02f0cb68f44969be535458d283f3e P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:18.332808 31887 raft_consensus.cc:2454] T 80e02f0cb68f44969be535458d283f3e P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:18.333259 29380 leader_election.cc:302] T 80e02f0cb68f44969be535458d283f3e P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dbcdc6ac635b44b185c3eba63f66e9d6, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:18.334286 1857 raft_consensus.cc:2781] T 80e02f0cb68f44969be535458d283f3e P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:18.334372 1857 raft_consensus.cc:686] T 80e02f0cb68f44969be535458d283f3e P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:18.334532 1857 consensus_queue.cc:237] T 80e02f0cb68f44969be535458d283f3e P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:18.339140 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e02f0cb68f44969be535458d283f3e" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:18.339290 31494 raft_consensus.cc:3037] T 80e02f0cb68f44969be535458d283f3e P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:18.340379 31494 raft_consensus.cc:2454] T 80e02f0cb68f44969be535458d283f3e P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:18.340611 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e02f0cb68f44969be535458d283f3e" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:18.340718 31495 raft_consensus.cc:2362] T 80e02f0cb68f44969be535458d283f3e P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1. Re-sending same reply.
I20240502 20:46:18.341398 29036 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:18.341526 29036 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:18.343016 29037 catalog_manager.cc:5484] T 80e02f0cb68f44969be535458d283f3e P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:18.343971 29036 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 773925f81a4147a6a0712ec84a2b3ff0
I20240502 20:46:18.344820 31735 tablet_service.cc:1514] Processing DeleteTablet for tablet 773925f81a4147a6a0712ec84a2b3ff0 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:18 UTC) from {username='slave'} at 127.0.0.1:33864
I20240502 20:46:18.345957 31475 tablet_service.cc:1514] Processing DeleteTablet for tablet 773925f81a4147a6a0712ec84a2b3ff0 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:18 UTC) from {username='slave'} at 127.0.0.1:40276
I20240502 20:46:18.346269 2046 tablet_replica.cc:330] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe: stopping tablet replica
I20240502 20:46:18.346423 2046 raft_consensus.cc:2229] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:18.346958 29557 tablet_service.cc:1514] Processing DeleteTablet for tablet 773925f81a4147a6a0712ec84a2b3ff0 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:18 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:18.347155 1859 tablet_replica.cc:330] T 773925f81a4147a6a0712ec84a2b3ff0 P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:18.347285 1859 raft_consensus.cc:2229] T 773925f81a4147a6a0712ec84a2b3ff0 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:18.347395 1859 raft_consensus.cc:2258] T 773925f81a4147a6a0712ec84a2b3ff0 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:18.350656 1859 ts_tablet_manager.cc:1875] T 773925f81a4147a6a0712ec84a2b3ff0 P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:18.352916 1859 ts_tablet_manager.cc:1888] T 773925f81a4147a6a0712ec84a2b3ff0 P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:18.353011 1859 log.cc:1196] T 773925f81a4147a6a0712ec84a2b3ff0 P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/773925f81a4147a6a0712ec84a2b3ff0
I20240502 20:46:18.353238 1859 ts_tablet_manager.cc:1909] T 773925f81a4147a6a0712ec84a2b3ff0 P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:18.353861 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet 773925f81a4147a6a0712ec84a2b3ff0 (table rebalancer_dynamic_table_new_name [id=808114c9773543fd8ad8c8c967fcb071]) successfully deleted
I20240502 20:46:18.346729 2046 raft_consensus.cc:2258] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:18.355516 2046 ts_tablet_manager.cc:1875] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:18.357668 2046 ts_tablet_manager.cc:1888] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:18.357748 2046 log.cc:1196] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/wals/773925f81a4147a6a0712ec84a2b3ff0
I20240502 20:46:18.358011 2046 ts_tablet_manager.cc:1909] T 773925f81a4147a6a0712ec84a2b3ff0 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting consensus metadata
I20240502 20:46:18.358637 2045 tablet_replica.cc:330] T 773925f81a4147a6a0712ec84a2b3ff0 P e7ab6b9053664ee5aa71c7978e26fe6a: stopping tablet replica
I20240502 20:46:18.358723 29028 catalog_manager.cc:4830] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): tablet 773925f81a4147a6a0712ec84a2b3ff0 (table rebalancer_dynamic_table_new_name [id=808114c9773543fd8ad8c8c967fcb071]) successfully deleted
I20240502 20:46:18.358850 2045 raft_consensus.cc:2229] T 773925f81a4147a6a0712ec84a2b3ff0 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:18.358989 2045 raft_consensus.cc:2258] T 773925f81a4147a6a0712ec84a2b3ff0 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:18.359589 2045 ts_tablet_manager.cc:1875] T 773925f81a4147a6a0712ec84a2b3ff0 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:18.361685 2045 ts_tablet_manager.cc:1888] T 773925f81a4147a6a0712ec84a2b3ff0 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:18.361754 2045 log.cc:1196] T 773925f81a4147a6a0712ec84a2b3ff0 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/wals/773925f81a4147a6a0712ec84a2b3ff0
I20240502 20:46:18.361966 2045 ts_tablet_manager.cc:1909] T 773925f81a4147a6a0712ec84a2b3ff0 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting consensus metadata
I20240502 20:46:18.362785 29026 catalog_manager.cc:4830] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): tablet 773925f81a4147a6a0712ec84a2b3ff0 (table rebalancer_dynamic_table_new_name [id=808114c9773543fd8ad8c8c967fcb071]) successfully deleted
I20240502 20:46:18.406409 692 raft_consensus.cc:482] T 46c0a5b0870c4a099c7c17ebc0a5d34e P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:18.406584 692 raft_consensus.cc:504] T 46c0a5b0870c4a099c7c17ebc0a5d34e P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:18.407045 692 leader_election.cc:288] T 46c0a5b0870c4a099c7c17ebc0a5d34e P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:18.407434 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46c0a5b0870c4a099c7c17ebc0a5d34e" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:18.407523 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46c0a5b0870c4a099c7c17ebc0a5d34e" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:18.407601 31626 raft_consensus.cc:2454] T 46c0a5b0870c4a099c7c17ebc0a5d34e P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:18.407655 29446 raft_consensus.cc:2454] T 46c0a5b0870c4a099c7c17ebc0a5d34e P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:18.407948 31692 leader_election.cc:302] T 46c0a5b0870c4a099c7c17ebc0a5d34e P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dbcdc6ac635b44b185c3eba63f66e9d6, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:18.408239 692 raft_consensus.cc:2781] T 46c0a5b0870c4a099c7c17ebc0a5d34e P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:18.408304 692 raft_consensus.cc:482] T 46c0a5b0870c4a099c7c17ebc0a5d34e P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:18.408349 692 raft_consensus.cc:3037] T 46c0a5b0870c4a099c7c17ebc0a5d34e P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:18.409299 692 raft_consensus.cc:504] T 46c0a5b0870c4a099c7c17ebc0a5d34e P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:18.409651 692 leader_election.cc:288] T 46c0a5b0870c4a099c7c17ebc0a5d34e P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:18.409973 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46c0a5b0870c4a099c7c17ebc0a5d34e" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:18.409998 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46c0a5b0870c4a099c7c17ebc0a5d34e" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:18.410085 31626 raft_consensus.cc:3037] T 46c0a5b0870c4a099c7c17ebc0a5d34e P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:18.410110 29446 raft_consensus.cc:3037] T 46c0a5b0870c4a099c7c17ebc0a5d34e P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:18.411020 31626 raft_consensus.cc:2454] T 46c0a5b0870c4a099c7c17ebc0a5d34e P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:18.411070 29446 raft_consensus.cc:2454] T 46c0a5b0870c4a099c7c17ebc0a5d34e P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:18.411351 31692 leader_election.cc:302] T 46c0a5b0870c4a099c7c17ebc0a5d34e P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dbcdc6ac635b44b185c3eba63f66e9d6, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:18.411545 692 raft_consensus.cc:2781] T 46c0a5b0870c4a099c7c17ebc0a5d34e P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:18.411643 692 raft_consensus.cc:686] T 46c0a5b0870c4a099c7c17ebc0a5d34e P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Becoming Leader. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Running, Role: LEADER
I20240502 20:46:18.411782 692 consensus_queue.cc:237] T 46c0a5b0870c4a099c7c17ebc0a5d34e P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:18.413652 29040 catalog_manager.cc:5484] T 46c0a5b0870c4a099c7c17ebc0a5d34e P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: term changed from 0 to 1, leader changed from <none> to e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71). New cstate: current_term: 1 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:18.437485 29162 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:18.442556 31475 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:18.442741 29293 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:18.447886 29557 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:18.448603 29424 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:18.449312 29040 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:18.455750 31475 tablet_service.cc:1467] Processing CreateTablet for tablet f107709bdafc4d24aca6f5c21cd265cf (DEFAULT_TABLE table=rebalancer_dynamic_table [id=98b6abad835643239e91d41d5a0fedec]), partition=
I20240502 20:46:18.455809 29557 tablet_service.cc:1467] Processing CreateTablet for tablet f107709bdafc4d24aca6f5c21cd265cf (DEFAULT_TABLE table=rebalancer_dynamic_table [id=98b6abad835643239e91d41d5a0fedec]), partition=
I20240502 20:46:18.456040 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f107709bdafc4d24aca6f5c21cd265cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:18.456034 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f107709bdafc4d24aca6f5c21cd265cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:18.456395 29293 tablet_service.cc:1467] Processing CreateTablet for tablet f107709bdafc4d24aca6f5c21cd265cf (DEFAULT_TABLE table=rebalancer_dynamic_table [id=98b6abad835643239e91d41d5a0fedec]), partition=
I20240502 20:46:18.456605 29293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f107709bdafc4d24aca6f5c21cd265cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:18.458216 1154 tablet_bootstrap.cc:492] T f107709bdafc4d24aca6f5c21cd265cf P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:18.458443 1752 tablet_bootstrap.cc:492] T f107709bdafc4d24aca6f5c21cd265cf P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:18.458688 31867 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:18.458900 2059 tablet_bootstrap.cc:492] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:18.459172 1154 tablet_bootstrap.cc:654] T f107709bdafc4d24aca6f5c21cd265cf P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:18.459352 1752 tablet_bootstrap.cc:654] T f107709bdafc4d24aca6f5c21cd265cf P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:18.461091 2059 tablet_bootstrap.cc:654] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:18.461143 1154 tablet_bootstrap.cc:492] T f107709bdafc4d24aca6f5c21cd265cf P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:18.461984 31602 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:18.461230 1154 ts_tablet_manager.cc:1367] T f107709bdafc4d24aca6f5c21cd265cf P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:18.463367 1154 raft_consensus.cc:348] T f107709bdafc4d24aca6f5c21cd265cf P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:18.463474 1154 raft_consensus.cc:374] T f107709bdafc4d24aca6f5c21cd265cf P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:18.463500 1154 raft_consensus.cc:729] T f107709bdafc4d24aca6f5c21cd265cf P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:18.463608 1154 consensus_queue.cc:260] T f107709bdafc4d24aca6f5c21cd265cf P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:18.463856 1154 ts_tablet_manager.cc:1398] T f107709bdafc4d24aca6f5c21cd265cf P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:18.465111 31735 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:18.465850 1752 tablet_bootstrap.cc:492] T f107709bdafc4d24aca6f5c21cd265cf P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:18.465940 1752 ts_tablet_manager.cc:1367] T f107709bdafc4d24aca6f5c21cd265cf P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20240502 20:46:18.466373 1752 raft_consensus.cc:348] T f107709bdafc4d24aca6f5c21cd265cf P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:18.466506 1752 raft_consensus.cc:374] T f107709bdafc4d24aca6f5c21cd265cf P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:18.466542 1752 raft_consensus.cc:729] T f107709bdafc4d24aca6f5c21cd265cf P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:18.466671 1752 consensus_queue.cc:260] T f107709bdafc4d24aca6f5c21cd265cf P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:18.466954 1752 ts_tablet_manager.cc:1398] T f107709bdafc4d24aca6f5c21cd265cf P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:18.497498 2059 tablet_bootstrap.cc:492] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:18.497612 2059 ts_tablet_manager.cc:1367] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.039s user 0.002s sys 0.000s
I20240502 20:46:18.498055 2059 raft_consensus.cc:348] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:18.498178 2059 raft_consensus.cc:374] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:18.498206 2059 raft_consensus.cc:729] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:18.498317 2059 consensus_queue.cc:260] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:18.498596 2059 ts_tablet_manager.cc:1398] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:18.528553 31887 consensus_queue.cc:237] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:18.529784 29314 raft_consensus.cc:1261] T 11ee3029cd744581ba9c5c213ca8fd2b P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:18.529973 31626 raft_consensus.cc:1261] T 11ee3029cd744581ba9c5c213ca8fd2b P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:18.530328 1844 consensus_queue.cc:1035] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:18.530644 1844 consensus_queue.cc:1035] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:18.532622 1844 raft_consensus.cc:2932] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:18.533038 29314 raft_consensus.cc:2932] T 11ee3029cd744581ba9c5c213ca8fd2b P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:18.535128 31626 raft_consensus.cc:2932] T 11ee3029cd744581ba9c5c213ca8fd2b P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:18.535209 29040 catalog_manager.cc:5484] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 reported cstate change: config changed from index 2 to 3, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) evicted. New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:18.540376 31735 tablet_service.cc:1514] Processing DeleteTablet for tablet 11ee3029cd744581ba9c5c213ca8fd2b with delete_type TABLET_DATA_TOMBSTONED (TS e7ab6b9053664ee5aa71c7978e26fe6a not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:33864
I20240502 20:46:18.540732 2045 tablet_replica.cc:330] T 11ee3029cd744581ba9c5c213ca8fd2b P e7ab6b9053664ee5aa71c7978e26fe6a: stopping tablet replica
I20240502 20:46:18.540890 2045 raft_consensus.cc:2229] T 11ee3029cd744581ba9c5c213ca8fd2b P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:18.541002 2045 raft_consensus.cc:2258] T 11ee3029cd744581ba9c5c213ca8fd2b P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:18.541383 2045 ts_tablet_manager.cc:1875] T 11ee3029cd744581ba9c5c213ca8fd2b P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:18.543115 2045 ts_tablet_manager.cc:1888] T 11ee3029cd744581ba9c5c213ca8fd2b P e7ab6b9053664ee5aa71c7978e26fe6a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 20:46:18.543167 2045 log.cc:1196] T 11ee3029cd744581ba9c5c213ca8fd2b P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/wals/11ee3029cd744581ba9c5c213ca8fd2b
I20240502 20:46:18.543634 29026 catalog_manager.cc:4830] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): tablet 11ee3029cd744581ba9c5c213ca8fd2b (table rebalancer_extra_table_10 [id=9bcf2b1be3a34dd7a9f3320c43834eda]) successfully deleted
I20240502 20:46:18.565156 29314 consensus_queue.cc:237] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:18.566715 29446 raft_consensus.cc:1261] T 689b7368353a430dbffa1cf44f9b5949 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 20:46:18.566809 29250 consensus_peers.cc:485] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13 -> Peer 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): Couldn't send request to peer 2a9c1af8cf814030bde56b67ca5c2fbe. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 689b7368353a430dbffa1cf44f9b5949. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:18.566819 29574 raft_consensus.cc:1261] T 689b7368353a430dbffa1cf44f9b5949 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:18.567179 2104 consensus_queue.cc:1035] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:18.567392 1525 consensus_queue.cc:1035] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:18.569146 1525 raft_consensus.cc:2932] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:18.569588 29574 raft_consensus.cc:2932] T 689b7368353a430dbffa1cf44f9b5949 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:18.571404 29446 raft_consensus.cc:2932] T 689b7368353a430dbffa1cf44f9b5949 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:18.572127 29037 catalog_manager.cc:5484] T 689b7368353a430dbffa1cf44f9b5949 P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:18.591574 31887 consensus_queue.cc:237] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:18.592917 31757 raft_consensus.cc:1261] T 075d3240e24a4532bbb29a068c00c128 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 20:46:18.593068 31822 consensus_peers.cc:485] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 -> Peer db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): Couldn't send request to peer db17f26a73f3447b92aa9a947e3a7057. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 075d3240e24a4532bbb29a068c00c128. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:18.593072 29182 raft_consensus.cc:1261] T 075d3240e24a4532bbb29a068c00c128 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:18.593338 1844 consensus_queue.cc:1035] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:18.593514 2082 consensus_queue.cc:1035] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:18.595273 1844 raft_consensus.cc:2932] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:18.595731 31757 raft_consensus.cc:2932] T 075d3240e24a4532bbb29a068c00c128 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:18.597951 29039 catalog_manager.cc:5484] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 reported cstate change: config changed from index -1 to 2, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) added. New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:18.598057 29182 raft_consensus.cc:2932] T 075d3240e24a4532bbb29a068c00c128 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:18.621766 29182 consensus_queue.cc:237] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:18.623211 29574 raft_consensus.cc:1261] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 20:46:18.623476 29118 consensus_peers.cc:485] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d -> Peer e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): Couldn't send request to peer e6b60803d0764d228c863e2da65e3691. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8c21a6762eaa4ba4ac1c91a42fb37849. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:18.623545 29446 raft_consensus.cc:1261] T 8c21a6762eaa4ba4ac1c91a42fb37849 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:18.623682 1776 consensus_queue.cc:1035] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:18.623961 1777 consensus_queue.cc:1035] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:18.626029 2143 raft_consensus.cc:2932] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:18.626451 29446 raft_consensus.cc:2932] T 8c21a6762eaa4ba4ac1c91a42fb37849 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:18.629226 29037 catalog_manager.cc:5484] T 8c21a6762eaa4ba4ac1c91a42fb37849 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:18.631299 29574 raft_consensus.cc:2932] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:18.653069 29574 consensus_queue.cc:237] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:18.654633 29446 raft_consensus.cc:1261] T fbd2bd8e142741ba864e2f99610b39d6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:18.654676 29182 raft_consensus.cc:1261] T fbd2bd8e142741ba864e2f99610b39d6 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 20:46:18.654703 29511 consensus_peers.cc:485] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fbd2bd8e142741ba864e2f99610b39d6. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:18.655059 2161 consensus_queue.cc:1035] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:18.655265 1688 consensus_queue.cc:1035] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:18.657078 1688 raft_consensus.cc:2932] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:18.657474 29182 raft_consensus.cc:2932] T fbd2bd8e142741ba864e2f99610b39d6 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:18.659610 29446 raft_consensus.cc:2932] T fbd2bd8e142741ba864e2f99610b39d6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:18.660087 29039 catalog_manager.cc:5484] T fbd2bd8e142741ba864e2f99610b39d6 P 7796ea3fba294d17a34386873969431d reported cstate change: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:18.685741 31757 consensus_queue.cc:237] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:18.687381 31495 raft_consensus.cc:1261] T f14cf6deefec46c7adb598f6d7cfd649 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Refusing update from remote peer e7ab6b9053664ee5aa71c7978e26fe6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 20:46:18.687788 31693 consensus_peers.cc:485] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): Couldn't send request to peer 7796ea3fba294d17a34386873969431d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f14cf6deefec46c7adb598f6d7cfd649. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:18.688323 29574 raft_consensus.cc:1261] T f14cf6deefec46c7adb598f6d7cfd649 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer e7ab6b9053664ee5aa71c7978e26fe6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:18.688197 2181 consensus_queue.cc:1035] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:18.688671 975 consensus_queue.cc:1035] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:18.690212 2181 raft_consensus.cc:2932] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:18.690634 29574 raft_consensus.cc:2932] T f14cf6deefec46c7adb598f6d7cfd649 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:18.692293 31495 raft_consensus.cc:2932] T f14cf6deefec46c7adb598f6d7cfd649 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:18.693043 29038 catalog_manager.cc:5484] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: config changed from index -1 to 2, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) added. New cstate: current_term: 1 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:18.704733 2164 consensus_queue.cc:1035] T e90d62851df5410481db24b0ab26a274 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:18.708117 1688 consensus_queue.cc:1035] T e90d62851df5410481db24b0ab26a274 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:18.719188 31626 consensus_queue.cc:237] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:18.720784 31887 raft_consensus.cc:1261] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer 315915efcf2b413ca7f9d874338e1b2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:18.720944 31757 raft_consensus.cc:1261] T 03db15f1ebfb4e4ea271d6bb5c76110d P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer 315915efcf2b413ca7f9d874338e1b2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 20:46:18.721163 31560 consensus_peers.cc:485] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d -> Peer db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): Couldn't send request to peer db17f26a73f3447b92aa9a947e3a7057. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 03db15f1ebfb4e4ea271d6bb5c76110d. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:18.721396 1055 consensus_queue.cc:1035] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:18.721702 1055 consensus_queue.cc:1035] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:18.727627 2205 raft_consensus.cc:2932] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:18.728049 31888 raft_consensus.cc:2932] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:18.730500 29037 catalog_manager.cc:5484] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 reported cstate change: config changed from index -1 to 2, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) added. New cstate: current_term: 1 leader_uuid: "315915efcf2b413ca7f9d874338e1b2d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:18.733036 31757 raft_consensus.cc:2932] T 03db15f1ebfb4e4ea271d6bb5c76110d P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:18.749104 2104 raft_consensus.cc:482] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:18.749238 2104 raft_consensus.cc:504] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:18.749686 2104 leader_election.cc:288] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:18.750017 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f107709bdafc4d24aca6f5c21cd265cf" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:18.750062 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f107709bdafc4d24aca6f5c21cd265cf" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:18.750181 31495 raft_consensus.cc:2454] T f107709bdafc4d24aca6f5c21cd265cf P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:18.750181 29574 raft_consensus.cc:2454] T f107709bdafc4d24aca6f5c21cd265cf P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:18.750540 29249 leader_election.cc:302] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:18.750767 2104 raft_consensus.cc:2781] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:18.750828 2104 raft_consensus.cc:482] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:18.750871 2104 raft_consensus.cc:3037] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:18.751120 29315 consensus_queue.cc:237] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:18.751956 2104 raft_consensus.cc:504] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:18.752259 2104 leader_election.cc:288] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:18.752746 29182 raft_consensus.cc:1261] T d4a2c7b197f948a0b236d5e05e2749b9 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:18.752712 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f107709bdafc4d24aca6f5c21cd265cf" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:18.752812 29446 raft_consensus.cc:1261] T d4a2c7b197f948a0b236d5e05e2749b9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:18.752831 31495 raft_consensus.cc:3037] T f107709bdafc4d24aca6f5c21cd265cf P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:18.753115 2224 consensus_queue.cc:1035] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:18.753755 31495 raft_consensus.cc:2454] T f107709bdafc4d24aca6f5c21cd265cf P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:18.754040 29250 leader_election.cc:302] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 2a9c1af8cf814030bde56b67ca5c2fbe; no voters:
I20240502 20:46:18.754201 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f107709bdafc4d24aca6f5c21cd265cf" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:18.754303 2224 raft_consensus.cc:2781] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:18.754317 29577 raft_consensus.cc:3037] T f107709bdafc4d24aca6f5c21cd265cf P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
W20240502 20:46:18.754344 29249 consensus_peers.cc:485] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d4a2c7b197f948a0b236d5e05e2749b9. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:18.754379 2224 raft_consensus.cc:686] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:18.754519 2224 consensus_queue.cc:237] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:18.754885 1996 consensus_queue.cc:1035] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:18.755329 29577 raft_consensus.cc:2454] T f107709bdafc4d24aca6f5c21cd265cf P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:18.756805 29039 catalog_manager.cc:5484] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:18.757580 2104 raft_consensus.cc:2932] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:18.758255 29446 raft_consensus.cc:2932] T d4a2c7b197f948a0b236d5e05e2749b9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:18.760061 29182 raft_consensus.cc:2932] T d4a2c7b197f948a0b236d5e05e2749b9 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:18.760929 29037 catalog_manager.cc:5484] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 reported cstate change: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:18.780318 31888 consensus_queue.cc:237] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:18.781672 31495 raft_consensus.cc:1261] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:18.781701 29182 raft_consensus.cc:1261] T 6b675480162c4a9799db51ea4b6504a5 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 20:46:18.781896 31823 consensus_peers.cc:485] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 -> Peer 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): Couldn't send request to peer 09258ac095664fa99caf0707262cde13. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6b675480162c4a9799db51ea4b6504a5. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:18.782102 1844 consensus_queue.cc:1035] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:18.782290 2079 consensus_queue.cc:1035] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:18.784062 1844 raft_consensus.cc:2932] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:18.784440 29182 raft_consensus.cc:2932] T 6b675480162c4a9799db51ea4b6504a5 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:18.785962 31495 raft_consensus.cc:2932] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:18.786909 29038 catalog_manager.cc:5484] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 reported cstate change: config changed from index -1 to 2, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) added. New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:18.805510 29446 consensus_queue.cc:237] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:18.806839 29315 raft_consensus.cc:1261] T a2bf196bcab440c48aa4ea893997573b P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 20:46:18.806953 29380 consensus_peers.cc:485] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): Couldn't send request to peer e6b60803d0764d228c863e2da65e3691. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a2bf196bcab440c48aa4ea893997573b. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:18.807068 29577 raft_consensus.cc:1261] T a2bf196bcab440c48aa4ea893997573b P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:18.807480 1857 consensus_queue.cc:1035] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:18.807688 2263 consensus_queue.cc:1035] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:18.809446 2263 raft_consensus.cc:2932] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:18.809890 29577 raft_consensus.cc:2932] T a2bf196bcab440c48aa4ea893997573b P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:18.812538 29039 catalog_manager.cc:5484] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:18.812583 29315 raft_consensus.cc:2932] T a2bf196bcab440c48aa4ea893997573b P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:18.817126 1321 consensus_queue.cc:1035] T 80e02f0cb68f44969be535458d283f3e P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:18.820387 1321 consensus_queue.cc:1035] T 80e02f0cb68f44969be535458d283f3e P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:18.835188 31625 consensus_queue.cc:237] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
W20240502 20:46:18.836673 31559 consensus_peers.cc:485] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d -> Peer e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): Couldn't send request to peer e7ab6b9053664ee5aa71c7978e26fe6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 58d4026c208f45d8870a2eaa1c97efab. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:18.836719 29182 raft_consensus.cc:1261] T 58d4026c208f45d8870a2eaa1c97efab P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer 315915efcf2b413ca7f9d874338e1b2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:18.837090 1055 consensus_queue.cc:1035] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:18.842437 29577 raft_consensus.cc:1261] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 315915efcf2b413ca7f9d874338e1b2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:18.842859 1055 consensus_queue.cc:1035] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:18.844564 2204 raft_consensus.cc:2932] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:18.845639 2184 consensus_queue.cc:1035] T 46c0a5b0870c4a099c7c17ebc0a5d34e P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:18.847287 29037 catalog_manager.cc:5484] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d reported cstate change: config changed from index -1 to 2, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New cstate: current_term: 1 leader_uuid: "315915efcf2b413ca7f9d874338e1b2d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:18.848309 29182 raft_consensus.cc:2932] T 58d4026c208f45d8870a2eaa1c97efab P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:18.850603 29577 raft_consensus.cc:2932] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:18.855634 2184 consensus_queue.cc:1035] T 46c0a5b0870c4a099c7c17ebc0a5d34e P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:18.872455 29038 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:18.876696 29446 consensus_queue.cc:237] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:18.877591 29577 raft_consensus.cc:1261] T f107709bdafc4d24aca6f5c21cd265cf P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:18.878055 2104 consensus_queue.cc:1035] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:18.878216 29182 raft_consensus.cc:1261] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:18.878275 29315 raft_consensus.cc:1261] T 4a91eafc5ecf45b09f60d712884b7c79 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:18.878667 31495 raft_consensus.cc:1261] T f107709bdafc4d24aca6f5c21cd265cf P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:18.878973 1321 consensus_queue.cc:1035] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:18.879094 2104 consensus_queue.cc:1035] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 20:46:18.879186 29380 consensus_peers.cc:485] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4a91eafc5ecf45b09f60d712884b7c79. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:18.879289 2263 consensus_queue.cc:1035] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:18.881531 29038 catalog_manager.cc:5576] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887) does not have the latest schema for tablet f107709bdafc4d24aca6f5c21cd265cf (table rebalancer_dynamic_table [id=98b6abad835643239e91d41d5a0fedec]). Expected version 1 got 0
I20240502 20:46:18.883136 29037 catalog_manager.cc:5576] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897) does not have the latest schema for tablet f107709bdafc4d24aca6f5c21cd265cf (table rebalancer_dynamic_table [id=98b6abad835643239e91d41d5a0fedec]). Expected version 1 got 0
I20240502 20:46:18.884253 2317 tablet.cc:1722] T f107709bdafc4d24aca6f5c21cd265cf P db17f26a73f3447b92aa9a947e3a7057: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:18.885195 29040 catalog_manager.cc:5576] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887) does not have the latest schema for tablet f107709bdafc4d24aca6f5c21cd265cf (table rebalancer_dynamic_table [id=98b6abad835643239e91d41d5a0fedec]). Expected version 1 got 0
I20240502 20:46:18.885205 2263 raft_consensus.cc:2932] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:18.885546 29037 catalog_manager.cc:5576] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897) does not have the latest schema for tablet f107709bdafc4d24aca6f5c21cd265cf (table rebalancer_dynamic_table [id=98b6abad835643239e91d41d5a0fedec]). Expected version 1 got 0
I20240502 20:46:18.887539 29315 raft_consensus.cc:2932] T 4a91eafc5ecf45b09f60d712884b7c79 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:18.888234 2315 tablet.cc:1722] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:18.888168 29182 raft_consensus.cc:2932] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:18.884634 2317 tablet.cc:1620] T f107709bdafc4d24aca6f5c21cd265cf P db17f26a73f3447b92aa9a947e3a7057: MemRowSet was empty: no flush needed.
I20240502 20:46:18.887883 2316 tablet.cc:1722] T f107709bdafc4d24aca6f5c21cd265cf P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:18.889058 2316 tablet.cc:1620] T f107709bdafc4d24aca6f5c21cd265cf P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:18.889344 2315 tablet.cc:1620] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13: MemRowSet was empty: no flush needed.
I20240502 20:46:18.890488 29040 catalog_manager.cc:5484] T 4a91eafc5ecf45b09f60d712884b7c79 P 09258ac095664fa99caf0707262cde13 reported cstate change: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:18.900065 29038 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=98b6abad835643239e91d41d5a0fedec] alter complete (version 1)
I20240502 20:46:18.974597 29036 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_18"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:18.983640 31475 tablet_service.cc:1467] Processing CreateTablet for tablet 1ce967c38f6f48c7a14b837335d1deca (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=399cd1fb43834edcbd567f6c776e5221]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:18.983714 31474 tablet_service.cc:1467] Processing CreateTablet for tablet fcf606426b8b44bcb660d6c2ed4af6e3 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=399cd1fb43834edcbd567f6c776e5221]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:18.983963 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ce967c38f6f48c7a14b837335d1deca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:18.984112 31474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcf606426b8b44bcb660d6c2ed4af6e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:18.984163 31602 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:18.984462 29293 tablet_service.cc:1467] Processing CreateTablet for tablet 696d094d24fe48a3aeeb01e3b15a50e4 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=399cd1fb43834edcbd567f6c776e5221]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:18.984755 29293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 696d094d24fe48a3aeeb01e3b15a50e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:18.986876 2373 tablet_bootstrap.cc:492] T fcf606426b8b44bcb660d6c2ed4af6e3 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:18.987612 31736 tablet_service.cc:1467] Processing CreateTablet for tablet 1ce967c38f6f48c7a14b837335d1deca (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=399cd1fb43834edcbd567f6c776e5221]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:18.987869 31736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ce967c38f6f48c7a14b837335d1deca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:18.987836 31737 tablet_service.cc:1467] Processing CreateTablet for tablet fcf606426b8b44bcb660d6c2ed4af6e3 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=399cd1fb43834edcbd567f6c776e5221]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:18.987898 29557 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:18.988016 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcf606426b8b44bcb660d6c2ed4af6e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:18.990028 2374 tablet_bootstrap.cc:492] T fcf606426b8b44bcb660d6c2ed4af6e3 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:18.990356 2373 tablet_bootstrap.cc:654] T fcf606426b8b44bcb660d6c2ed4af6e3 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:18.987612 31735 tablet_service.cc:1467] Processing CreateTablet for tablet 696d094d24fe48a3aeeb01e3b15a50e4 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=399cd1fb43834edcbd567f6c776e5221]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:18.990702 31868 tablet_service.cc:1467] Processing CreateTablet for tablet fcf606426b8b44bcb660d6c2ed4af6e3 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=399cd1fb43834edcbd567f6c776e5221]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:18.990948 31735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 696d094d24fe48a3aeeb01e3b15a50e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:18.990970 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcf606426b8b44bcb660d6c2ed4af6e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:18.990927 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 1ce967c38f6f48c7a14b837335d1deca (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=399cd1fb43834edcbd567f6c776e5221]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:18.991118 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ce967c38f6f48c7a14b837335d1deca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:18.991822 2374 tablet_bootstrap.cc:654] T fcf606426b8b44bcb660d6c2ed4af6e3 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:18.993005 2374 tablet_bootstrap.cc:492] T fcf606426b8b44bcb660d6c2ed4af6e3 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:18.993077 2374 ts_tablet_manager.cc:1367] T fcf606426b8b44bcb660d6c2ed4af6e3 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:18.993530 2374 raft_consensus.cc:348] T fcf606426b8b44bcb660d6c2ed4af6e3 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:18.993630 2374 raft_consensus.cc:374] T fcf606426b8b44bcb660d6c2ed4af6e3 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:18.993656 2374 raft_consensus.cc:729] T fcf606426b8b44bcb660d6c2ed4af6e3 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:18.993757 2374 consensus_queue.cc:260] T fcf606426b8b44bcb660d6c2ed4af6e3 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:18.994004 2374 ts_tablet_manager.cc:1398] T fcf606426b8b44bcb660d6c2ed4af6e3 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:18.994144 2374 tablet_bootstrap.cc:492] T 696d094d24fe48a3aeeb01e3b15a50e4 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:18.994659 29162 tablet_service.cc:1467] Processing CreateTablet for tablet 696d094d24fe48a3aeeb01e3b15a50e4 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=399cd1fb43834edcbd567f6c776e5221]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:18.994946 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 696d094d24fe48a3aeeb01e3b15a50e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:18.994973 2374 tablet_bootstrap.cc:654] T 696d094d24fe48a3aeeb01e3b15a50e4 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:18.997330 2374 tablet_bootstrap.cc:492] T 696d094d24fe48a3aeeb01e3b15a50e4 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:18.997412 2374 ts_tablet_manager.cc:1367] T 696d094d24fe48a3aeeb01e3b15a50e4 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:18.997849 2374 raft_consensus.cc:348] T 696d094d24fe48a3aeeb01e3b15a50e4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:18.997941 2374 raft_consensus.cc:374] T 696d094d24fe48a3aeeb01e3b15a50e4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:18.997968 2374 raft_consensus.cc:729] T 696d094d24fe48a3aeeb01e3b15a50e4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:18.998075 2374 consensus_queue.cc:260] T 696d094d24fe48a3aeeb01e3b15a50e4 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:18.998354 2374 ts_tablet_manager.cc:1398] T 696d094d24fe48a3aeeb01e3b15a50e4 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:18.999605 31735 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:19.000010 2379 tablet_bootstrap.cc:492] T 696d094d24fe48a3aeeb01e3b15a50e4 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:19.000967 2379 tablet_bootstrap.cc:654] T 696d094d24fe48a3aeeb01e3b15a50e4 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:19.002815 29295 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:19.005211 29040 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:19.006719 31475 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:19.007421 2374 tablet_bootstrap.cc:492] T 1ce967c38f6f48c7a14b837335d1deca P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:19.008370 2374 tablet_bootstrap.cc:654] T 1ce967c38f6f48c7a14b837335d1deca P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:19.010025 29424 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:19.010056 2377 tablet_bootstrap.cc:492] T 1ce967c38f6f48c7a14b837335d1deca P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:19.012595 31868 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:19.016129 2315 tablet.cc:1722] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:19.016441 2315 tablet.cc:1620] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13: MemRowSet was empty: no flush needed.
I20240502 20:46:19.019300 2317 tablet.cc:1722] T f107709bdafc4d24aca6f5c21cd265cf P db17f26a73f3447b92aa9a947e3a7057: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:19.019595 2317 tablet.cc:1620] T f107709bdafc4d24aca6f5c21cd265cf P db17f26a73f3447b92aa9a947e3a7057: MemRowSet was empty: no flush needed.
I20240502 20:46:19.026713 29162 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:19.026790 2316 tablet.cc:1722] T f107709bdafc4d24aca6f5c21cd265cf P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:19.027107 2316 tablet.cc:1620] T f107709bdafc4d24aca6f5c21cd265cf P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:19.027163 29040 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=98b6abad835643239e91d41d5a0fedec] alter complete (version 2)
I20240502 20:46:19.048202 2377 tablet_bootstrap.cc:654] T 1ce967c38f6f48c7a14b837335d1deca P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:19.050874 2381 tablet_bootstrap.cc:492] T 696d094d24fe48a3aeeb01e3b15a50e4 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:19.051893 2381 tablet_bootstrap.cc:654] T 696d094d24fe48a3aeeb01e3b15a50e4 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:19.051903 2374 tablet_bootstrap.cc:492] T 1ce967c38f6f48c7a14b837335d1deca P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:19.052006 2374 ts_tablet_manager.cc:1367] T 1ce967c38f6f48c7a14b837335d1deca P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.045s user 0.002s sys 0.000s
I20240502 20:46:19.052415 2374 raft_consensus.cc:348] T 1ce967c38f6f48c7a14b837335d1deca P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:19.052515 2374 raft_consensus.cc:374] T 1ce967c38f6f48c7a14b837335d1deca P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:19.052533 2374 raft_consensus.cc:729] T 1ce967c38f6f48c7a14b837335d1deca P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.052618 2374 consensus_queue.cc:260] T 1ce967c38f6f48c7a14b837335d1deca P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:19.052866 2374 ts_tablet_manager.cc:1398] T 1ce967c38f6f48c7a14b837335d1deca P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:19.058702 2386 ts_tablet_manager.cc:902] T fbd2bd8e142741ba864e2f99610b39d6 P 315915efcf2b413ca7f9d874338e1b2d: Initiating tablet copy from peer db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:19.059269 2386 tablet_copy_client.cc:308] T fbd2bd8e142741ba864e2f99610b39d6 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.68:45897
I20240502 20:46:19.059737 29597 tablet_copy_service.cc:140] P db17f26a73f3447b92aa9a947e3a7057: Received BeginTabletCopySession request for tablet fbd2bd8e142741ba864e2f99610b39d6 from peer 315915efcf2b413ca7f9d874338e1b2d ({username='slave'} at 127.22.45.70:48497)
I20240502 20:46:19.059834 29597 tablet_copy_service.cc:161] P db17f26a73f3447b92aa9a947e3a7057: Beginning new tablet copy session on tablet fbd2bd8e142741ba864e2f99610b39d6 from peer 315915efcf2b413ca7f9d874338e1b2d at {username='slave'} at 127.22.45.70:48497: session id = 315915efcf2b413ca7f9d874338e1b2d-fbd2bd8e142741ba864e2f99610b39d6
I20240502 20:46:19.060593 29597 tablet_copy_source_session.cc:205] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:19.061501 2386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbd2bd8e142741ba864e2f99610b39d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:19.062534 2377 tablet_bootstrap.cc:492] T 1ce967c38f6f48c7a14b837335d1deca P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:19.062706 2377 ts_tablet_manager.cc:1367] T 1ce967c38f6f48c7a14b837335d1deca P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.053s user 0.002s sys 0.000s
I20240502 20:46:19.063774 2377 raft_consensus.cc:348] T 1ce967c38f6f48c7a14b837335d1deca P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:19.063947 2377 raft_consensus.cc:374] T 1ce967c38f6f48c7a14b837335d1deca P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:19.064010 2377 raft_consensus.cc:729] T 1ce967c38f6f48c7a14b837335d1deca P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.064071 2386 tablet_copy_client.cc:783] T fbd2bd8e142741ba864e2f99610b39d6 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:19.064193 2377 consensus_queue.cc:260] T 1ce967c38f6f48c7a14b837335d1deca P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:19.064301 2386 tablet_copy_client.cc:647] T fbd2bd8e142741ba864e2f99610b39d6 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:19.064785 2377 ts_tablet_manager.cc:1398] T 1ce967c38f6f48c7a14b837335d1deca P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:19.064993 2377 tablet_bootstrap.cc:492] T fcf606426b8b44bcb660d6c2ed4af6e3 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:19.065894 2377 tablet_bootstrap.cc:654] T fcf606426b8b44bcb660d6c2ed4af6e3 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:19.065948 2386 tablet_copy_client.cc:523] T fbd2bd8e142741ba864e2f99610b39d6 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:19.067895 2382 ts_tablet_manager.cc:902] T 689b7368353a430dbffa1cf44f9b5949 P 2a9c1af8cf814030bde56b67ca5c2fbe: Initiating tablet copy from peer 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:19.068236 2386 tablet_bootstrap.cc:492] T fbd2bd8e142741ba864e2f99610b39d6 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:19.068457 2382 tablet_copy_client.cc:308] T 689b7368353a430dbffa1cf44f9b5949 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.66:40945
I20240502 20:46:19.068894 29335 tablet_copy_service.cc:140] P 09258ac095664fa99caf0707262cde13: Received BeginTabletCopySession request for tablet 689b7368353a430dbffa1cf44f9b5949 from peer 2a9c1af8cf814030bde56b67ca5c2fbe ({username='slave'} at 127.22.45.69:52445)
I20240502 20:46:19.068997 29335 tablet_copy_service.cc:161] P 09258ac095664fa99caf0707262cde13: Beginning new tablet copy session on tablet 689b7368353a430dbffa1cf44f9b5949 from peer 2a9c1af8cf814030bde56b67ca5c2fbe at {username='slave'} at 127.22.45.69:52445: session id = 2a9c1af8cf814030bde56b67ca5c2fbe-689b7368353a430dbffa1cf44f9b5949
I20240502 20:46:19.069904 29335 tablet_copy_source_session.cc:205] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:19.070804 2382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 689b7368353a430dbffa1cf44f9b5949. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:19.071146 2379 tablet_bootstrap.cc:492] T 696d094d24fe48a3aeeb01e3b15a50e4 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:19.071228 2379 ts_tablet_manager.cc:1367] T 696d094d24fe48a3aeeb01e3b15a50e4 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.071s user 0.002s sys 0.000s
I20240502 20:46:19.071596 2379 raft_consensus.cc:348] T 696d094d24fe48a3aeeb01e3b15a50e4 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:19.071686 2379 raft_consensus.cc:374] T 696d094d24fe48a3aeeb01e3b15a50e4 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:19.071705 2379 raft_consensus.cc:729] T 696d094d24fe48a3aeeb01e3b15a50e4 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.071811 2379 consensus_queue.cc:260] T 696d094d24fe48a3aeeb01e3b15a50e4 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:19.072067 2379 ts_tablet_manager.cc:1398] T 696d094d24fe48a3aeeb01e3b15a50e4 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:19.073185 2381 tablet_bootstrap.cc:492] T 696d094d24fe48a3aeeb01e3b15a50e4 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:19.073334 2381 ts_tablet_manager.cc:1367] T 696d094d24fe48a3aeeb01e3b15a50e4 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.000s
I20240502 20:46:19.073369 2382 tablet_copy_client.cc:783] T 689b7368353a430dbffa1cf44f9b5949 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:19.073513 2382 tablet_copy_client.cc:647] T 689b7368353a430dbffa1cf44f9b5949 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:19.074751 2381 raft_consensus.cc:348] T 696d094d24fe48a3aeeb01e3b15a50e4 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:19.074926 2381 raft_consensus.cc:374] T 696d094d24fe48a3aeeb01e3b15a50e4 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:19.075006 2381 raft_consensus.cc:729] T 696d094d24fe48a3aeeb01e3b15a50e4 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.075135 2398 ts_tablet_manager.cc:902] T f14cf6deefec46c7adb598f6d7cfd649 P 7796ea3fba294d17a34386873969431d: Initiating tablet copy from peer e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:19.075263 2381 consensus_queue.cc:260] T 696d094d24fe48a3aeeb01e3b15a50e4 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:19.075707 2381 ts_tablet_manager.cc:1398] T 696d094d24fe48a3aeeb01e3b15a50e4 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:19.075731 2382 tablet_copy_client.cc:523] T 689b7368353a430dbffa1cf44f9b5949 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:19.077039 2398 tablet_copy_client.cc:308] T f14cf6deefec46c7adb598f6d7cfd649 P 7796ea3fba294d17a34386873969431d: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.71:36787
I20240502 20:46:19.077378 2382 tablet_bootstrap.cc:492] T 689b7368353a430dbffa1cf44f9b5949 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:19.077725 31777 tablet_copy_service.cc:140] P e7ab6b9053664ee5aa71c7978e26fe6a: Received BeginTabletCopySession request for tablet f14cf6deefec46c7adb598f6d7cfd649 from peer 7796ea3fba294d17a34386873969431d ({username='slave'} at 127.22.45.65:43935)
I20240502 20:46:19.077849 31777 tablet_copy_service.cc:161] P e7ab6b9053664ee5aa71c7978e26fe6a: Beginning new tablet copy session on tablet f14cf6deefec46c7adb598f6d7cfd649 from peer 7796ea3fba294d17a34386873969431d at {username='slave'} at 127.22.45.65:43935: session id = 7796ea3fba294d17a34386873969431d-f14cf6deefec46c7adb598f6d7cfd649
I20240502 20:46:19.079404 31777 tablet_copy_source_session.cc:205] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:19.079764 2373 tablet_bootstrap.cc:492] T fcf606426b8b44bcb660d6c2ed4af6e3 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:19.079845 2373 ts_tablet_manager.cc:1367] T fcf606426b8b44bcb660d6c2ed4af6e3 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.093s user 0.002s sys 0.000s
I20240502 20:46:19.080004 2377 tablet_bootstrap.cc:492] T fcf606426b8b44bcb660d6c2ed4af6e3 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:19.080083 2377 ts_tablet_manager.cc:1367] T fcf606426b8b44bcb660d6c2ed4af6e3 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 20:46:19.080329 2373 raft_consensus.cc:348] T fcf606426b8b44bcb660d6c2ed4af6e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:19.080430 2373 raft_consensus.cc:374] T fcf606426b8b44bcb660d6c2ed4af6e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:19.080459 2373 raft_consensus.cc:729] T fcf606426b8b44bcb660d6c2ed4af6e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.080518 2377 raft_consensus.cc:348] T fcf606426b8b44bcb660d6c2ed4af6e3 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:19.080576 2373 consensus_queue.cc:260] T fcf606426b8b44bcb660d6c2ed4af6e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:19.080636 2377 raft_consensus.cc:374] T fcf606426b8b44bcb660d6c2ed4af6e3 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:19.080667 2377 raft_consensus.cc:729] T fcf606426b8b44bcb660d6c2ed4af6e3 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.080806 2377 consensus_queue.cc:260] T fcf606426b8b44bcb660d6c2ed4af6e3 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:19.080863 2373 ts_tablet_manager.cc:1398] T fcf606426b8b44bcb660d6c2ed4af6e3 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:19.081025 2373 tablet_bootstrap.cc:492] T 1ce967c38f6f48c7a14b837335d1deca P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:19.081895 2373 tablet_bootstrap.cc:654] T 1ce967c38f6f48c7a14b837335d1deca P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:19.083076 2398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f14cf6deefec46c7adb598f6d7cfd649. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:19.084035 2373 tablet_bootstrap.cc:492] T 1ce967c38f6f48c7a14b837335d1deca P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:19.084124 2373 ts_tablet_manager.cc:1367] T 1ce967c38f6f48c7a14b837335d1deca P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:19.084602 2373 raft_consensus.cc:348] T 1ce967c38f6f48c7a14b837335d1deca P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:19.084717 2373 raft_consensus.cc:374] T 1ce967c38f6f48c7a14b837335d1deca P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:19.084745 2373 raft_consensus.cc:729] T 1ce967c38f6f48c7a14b837335d1deca P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.085589 2373 consensus_queue.cc:260] T 1ce967c38f6f48c7a14b837335d1deca P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:19.086052 2373 ts_tablet_manager.cc:1398] T 1ce967c38f6f48c7a14b837335d1deca P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:19.086005 2398 tablet_copy_client.cc:783] T f14cf6deefec46c7adb598f6d7cfd649 P 7796ea3fba294d17a34386873969431d: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:19.086898 2398 tablet_copy_client.cc:647] T f14cf6deefec46c7adb598f6d7cfd649 P 7796ea3fba294d17a34386873969431d: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:19.087949 2377 ts_tablet_manager.cc:1398] T fcf606426b8b44bcb660d6c2ed4af6e3 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20240502 20:46:19.093068 2398 tablet_copy_client.cc:523] T f14cf6deefec46c7adb598f6d7cfd649 P 7796ea3fba294d17a34386873969431d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:19.094816 2398 tablet_bootstrap.cc:492] T f14cf6deefec46c7adb598f6d7cfd649 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:19.097903 2412 ts_tablet_manager.cc:902] T 03db15f1ebfb4e4ea271d6bb5c76110d P db17f26a73f3447b92aa9a947e3a7057: Initiating tablet copy from peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:19.098664 2412 tablet_copy_client.cc:308] T 03db15f1ebfb4e4ea271d6bb5c76110d P db17f26a73f3447b92aa9a947e3a7057: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.70:46751
I20240502 20:46:19.102727 31646 tablet_copy_service.cc:140] P 315915efcf2b413ca7f9d874338e1b2d: Received BeginTabletCopySession request for tablet 03db15f1ebfb4e4ea271d6bb5c76110d from peer db17f26a73f3447b92aa9a947e3a7057 ({username='slave'} at 127.22.45.68:55467)
I20240502 20:46:19.102865 31646 tablet_copy_service.cc:161] P 315915efcf2b413ca7f9d874338e1b2d: Beginning new tablet copy session on tablet 03db15f1ebfb4e4ea271d6bb5c76110d from peer db17f26a73f3447b92aa9a947e3a7057 at {username='slave'} at 127.22.45.68:55467: session id = db17f26a73f3447b92aa9a947e3a7057-03db15f1ebfb4e4ea271d6bb5c76110d
I20240502 20:46:19.104207 31646 tablet_copy_source_session.cc:205] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:19.105633 2412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03db15f1ebfb4e4ea271d6bb5c76110d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:19.108186 2412 tablet_copy_client.cc:783] T 03db15f1ebfb4e4ea271d6bb5c76110d P db17f26a73f3447b92aa9a947e3a7057: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:19.108383 2412 tablet_copy_client.cc:647] T 03db15f1ebfb4e4ea271d6bb5c76110d P db17f26a73f3447b92aa9a947e3a7057: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:19.110997 2417 ts_tablet_manager.cc:902] T 075d3240e24a4532bbb29a068c00c128 P db17f26a73f3447b92aa9a947e3a7057: Initiating tablet copy from peer e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:19.111647 2417 tablet_copy_client.cc:308] T 075d3240e24a4532bbb29a068c00c128 P db17f26a73f3447b92aa9a947e3a7057: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.72:35207
I20240502 20:46:19.112293 31908 tablet_copy_service.cc:140] P e6b60803d0764d228c863e2da65e3691: Received BeginTabletCopySession request for tablet 075d3240e24a4532bbb29a068c00c128 from peer db17f26a73f3447b92aa9a947e3a7057 ({username='slave'} at 127.22.45.68:38379)
I20240502 20:46:19.112404 31908 tablet_copy_service.cc:161] P e6b60803d0764d228c863e2da65e3691: Beginning new tablet copy session on tablet 075d3240e24a4532bbb29a068c00c128 from peer db17f26a73f3447b92aa9a947e3a7057 at {username='slave'} at 127.22.45.68:38379: session id = db17f26a73f3447b92aa9a947e3a7057-075d3240e24a4532bbb29a068c00c128
I20240502 20:46:19.113329 31908 tablet_copy_source_session.cc:205] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:19.113673 2398 tablet_bootstrap.cc:492] T f14cf6deefec46c7adb598f6d7cfd649 P 7796ea3fba294d17a34386873969431d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:19.113793 2184 raft_consensus.cc:482] T 1ce967c38f6f48c7a14b837335d1deca P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:19.113893 2184 raft_consensus.cc:504] T 1ce967c38f6f48c7a14b837335d1deca P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:19.113956 2398 tablet_bootstrap.cc:492] T f14cf6deefec46c7adb598f6d7cfd649 P 7796ea3fba294d17a34386873969431d: Bootstrap complete.
I20240502 20:46:19.114111 2398 ts_tablet_manager.cc:1367] T f14cf6deefec46c7adb598f6d7cfd649 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.000s
I20240502 20:46:19.114290 2184 leader_election.cc:288] T 1ce967c38f6f48c7a14b837335d1deca P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:19.114593 2398 raft_consensus.cc:348] T f14cf6deefec46c7adb598f6d7cfd649 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:19.114701 2398 raft_consensus.cc:729] T f14cf6deefec46c7adb598f6d7cfd649 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.114812 2398 consensus_queue.cc:260] T f14cf6deefec46c7adb598f6d7cfd649 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:19.114821 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ce967c38f6f48c7a14b837335d1deca" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:19.114953 31495 raft_consensus.cc:2454] T 1ce967c38f6f48c7a14b837335d1deca P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:19.115064 2398 ts_tablet_manager.cc:1398] T f14cf6deefec46c7adb598f6d7cfd649 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:19.115749 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ce967c38f6f48c7a14b837335d1deca" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:19.115870 31888 raft_consensus.cc:2454] T 1ce967c38f6f48c7a14b837335d1deca P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:19.116353 31777 tablet_copy_service.cc:342] P e7ab6b9053664ee5aa71c7978e26fe6a: Request end of tablet copy session 7796ea3fba294d17a34386873969431d-f14cf6deefec46c7adb598f6d7cfd649 received from {username='slave'} at 127.22.45.65:43935
I20240502 20:46:19.116436 31777 tablet_copy_service.cc:433] P e7ab6b9053664ee5aa71c7978e26fe6a: ending tablet copy session 7796ea3fba294d17a34386873969431d-f14cf6deefec46c7adb598f6d7cfd649 on tablet f14cf6deefec46c7adb598f6d7cfd649 with peer 7796ea3fba294d17a34386873969431d
I20240502 20:46:19.117274 31691 leader_election.cc:302] T 1ce967c38f6f48c7a14b837335d1deca P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e6b60803d0764d228c863e2da65e3691, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:19.117717 2184 raft_consensus.cc:2781] T 1ce967c38f6f48c7a14b837335d1deca P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:19.117774 2184 raft_consensus.cc:482] T 1ce967c38f6f48c7a14b837335d1deca P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:19.117810 2184 raft_consensus.cc:3037] T 1ce967c38f6f48c7a14b837335d1deca P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:19.118852 2184 raft_consensus.cc:504] T 1ce967c38f6f48c7a14b837335d1deca P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:19.119201 2184 leader_election.cc:288] T 1ce967c38f6f48c7a14b837335d1deca P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Requested vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:19.120612 2412 tablet_copy_client.cc:523] T 03db15f1ebfb4e4ea271d6bb5c76110d P db17f26a73f3447b92aa9a947e3a7057: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:19.120946 2386 tablet_bootstrap.cc:492] T fbd2bd8e142741ba864e2f99610b39d6 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:19.121120 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ce967c38f6f48c7a14b837335d1deca" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:19.121224 2386 tablet_bootstrap.cc:492] T fbd2bd8e142741ba864e2f99610b39d6 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap complete.
I20240502 20:46:19.121229 31888 raft_consensus.cc:3037] T 1ce967c38f6f48c7a14b837335d1deca P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:19.121480 2386 ts_tablet_manager.cc:1367] T fbd2bd8e142741ba864e2f99610b39d6 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.053s user 0.026s sys 0.000s
I20240502 20:46:19.121950 2386 raft_consensus.cc:348] T fbd2bd8e142741ba864e2f99610b39d6 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:19.122054 2386 raft_consensus.cc:729] T fbd2bd8e142741ba864e2f99610b39d6 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.122213 31888 raft_consensus.cc:2454] T 1ce967c38f6f48c7a14b837335d1deca P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:19.122200 2386 consensus_queue.cc:260] T fbd2bd8e142741ba864e2f99610b39d6 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:19.122354 2412 tablet_bootstrap.cc:492] T 03db15f1ebfb4e4ea271d6bb5c76110d P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:19.122570 2386 ts_tablet_manager.cc:1398] T fbd2bd8e142741ba864e2f99610b39d6 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:19.122936 31691 leader_election.cc:302] T 1ce967c38f6f48c7a14b837335d1deca P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e6b60803d0764d228c863e2da65e3691, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:19.122964 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ce967c38f6f48c7a14b837335d1deca" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:19.123076 31495 raft_consensus.cc:3037] T 1ce967c38f6f48c7a14b837335d1deca P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:19.123103 2184 raft_consensus.cc:2781] T 1ce967c38f6f48c7a14b837335d1deca P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:19.123096 29597 tablet_copy_service.cc:342] P db17f26a73f3447b92aa9a947e3a7057: Request end of tablet copy session 315915efcf2b413ca7f9d874338e1b2d-fbd2bd8e142741ba864e2f99610b39d6 received from {username='slave'} at 127.22.45.70:48497
I20240502 20:46:19.123160 29597 tablet_copy_service.cc:433] P db17f26a73f3447b92aa9a947e3a7057: ending tablet copy session 315915efcf2b413ca7f9d874338e1b2d-fbd2bd8e142741ba864e2f99610b39d6 on tablet fbd2bd8e142741ba864e2f99610b39d6 with peer 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:19.123170 2184 raft_consensus.cc:686] T 1ce967c38f6f48c7a14b837335d1deca P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Becoming Leader. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Running, Role: LEADER
I20240502 20:46:19.123291 2184 consensus_queue.cc:237] T 1ce967c38f6f48c7a14b837335d1deca P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:19.124064 31495 raft_consensus.cc:2454] T 1ce967c38f6f48c7a14b837335d1deca P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:19.127116 2422 ts_tablet_manager.cc:902] T 8c21a6762eaa4ba4ac1c91a42fb37849 P e6b60803d0764d228c863e2da65e3691: Initiating tablet copy from peer 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:19.127140 29040 catalog_manager.cc:5484] T 1ce967c38f6f48c7a14b837335d1deca P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: term changed from 0 to 1, leader changed from <none> to e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71). New cstate: current_term: 1 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:19.127791 2417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 075d3240e24a4532bbb29a068c00c128. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:19.128234 2422 tablet_copy_client.cc:308] T 8c21a6762eaa4ba4ac1c91a42fb37849 P e6b60803d0764d228c863e2da65e3691: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.65:43267
I20240502 20:46:19.129525 29202 tablet_copy_service.cc:140] P 7796ea3fba294d17a34386873969431d: Received BeginTabletCopySession request for tablet 8c21a6762eaa4ba4ac1c91a42fb37849 from peer e6b60803d0764d228c863e2da65e3691 ({username='slave'} at 127.22.45.72:55003)
I20240502 20:46:19.129628 29202 tablet_copy_service.cc:161] P 7796ea3fba294d17a34386873969431d: Beginning new tablet copy session on tablet 8c21a6762eaa4ba4ac1c91a42fb37849 from peer e6b60803d0764d228c863e2da65e3691 at {username='slave'} at 127.22.45.72:55003: session id = e6b60803d0764d228c863e2da65e3691-8c21a6762eaa4ba4ac1c91a42fb37849
I20240502 20:46:19.130554 2382 tablet_bootstrap.cc:492] T 689b7368353a430dbffa1cf44f9b5949 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:19.130585 29202 tablet_copy_source_session.cc:205] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:19.130853 2382 tablet_bootstrap.cc:492] T 689b7368353a430dbffa1cf44f9b5949 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap complete.
I20240502 20:46:19.131031 2382 ts_tablet_manager.cc:1367] T 689b7368353a430dbffa1cf44f9b5949 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.054s user 0.026s sys 0.000s
I20240502 20:46:19.131511 2422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c21a6762eaa4ba4ac1c91a42fb37849. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:19.131481 2382 raft_consensus.cc:348] T 689b7368353a430dbffa1cf44f9b5949 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:19.131584 2382 raft_consensus.cc:729] T 689b7368353a430dbffa1cf44f9b5949 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.131642 2417 tablet_copy_client.cc:783] T 075d3240e24a4532bbb29a068c00c128 P db17f26a73f3447b92aa9a947e3a7057: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:19.131786 2417 tablet_copy_client.cc:647] T 075d3240e24a4532bbb29a068c00c128 P db17f26a73f3447b92aa9a947e3a7057: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:19.131752 2382 consensus_queue.cc:260] T 689b7368353a430dbffa1cf44f9b5949 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:19.132040 2382 ts_tablet_manager.cc:1398] T 689b7368353a430dbffa1cf44f9b5949 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:19.132761 2412 tablet_bootstrap.cc:492] T 03db15f1ebfb4e4ea271d6bb5c76110d P db17f26a73f3447b92aa9a947e3a7057: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:19.133001 2412 tablet_bootstrap.cc:492] T 03db15f1ebfb4e4ea271d6bb5c76110d P db17f26a73f3447b92aa9a947e3a7057: Bootstrap complete.
I20240502 20:46:19.133251 2412 ts_tablet_manager.cc:1367] T 03db15f1ebfb4e4ea271d6bb5c76110d P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20240502 20:46:19.133634 2422 tablet_copy_client.cc:783] T 8c21a6762eaa4ba4ac1c91a42fb37849 P e6b60803d0764d228c863e2da65e3691: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:19.133668 2412 raft_consensus.cc:348] T 03db15f1ebfb4e4ea271d6bb5c76110d P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:19.133766 2412 raft_consensus.cc:729] T 03db15f1ebfb4e4ea271d6bb5c76110d P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.133882 2412 consensus_queue.cc:260] T 03db15f1ebfb4e4ea271d6bb5c76110d P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:19.134169 29335 tablet_copy_service.cc:342] P 09258ac095664fa99caf0707262cde13: Request end of tablet copy session 2a9c1af8cf814030bde56b67ca5c2fbe-689b7368353a430dbffa1cf44f9b5949 received from {username='slave'} at 127.22.45.69:52445
I20240502 20:46:19.134212 2412 ts_tablet_manager.cc:1398] T 03db15f1ebfb4e4ea271d6bb5c76110d P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:19.134241 29335 tablet_copy_service.cc:433] P 09258ac095664fa99caf0707262cde13: ending tablet copy session 2a9c1af8cf814030bde56b67ca5c2fbe-689b7368353a430dbffa1cf44f9b5949 on tablet 689b7368353a430dbffa1cf44f9b5949 with peer 2a9c1af8cf814030bde56b67ca5c2fbe
I20240502 20:46:19.136152 31646 tablet_copy_service.cc:342] P 315915efcf2b413ca7f9d874338e1b2d: Request end of tablet copy session db17f26a73f3447b92aa9a947e3a7057-03db15f1ebfb4e4ea271d6bb5c76110d received from {username='slave'} at 127.22.45.68:55467
I20240502 20:46:19.136262 31646 tablet_copy_service.cc:433] P 315915efcf2b413ca7f9d874338e1b2d: ending tablet copy session db17f26a73f3447b92aa9a947e3a7057-03db15f1ebfb4e4ea271d6bb5c76110d on tablet 03db15f1ebfb4e4ea271d6bb5c76110d with peer db17f26a73f3447b92aa9a947e3a7057
I20240502 20:46:19.136933 29035 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:19.137550 2422 tablet_copy_client.cc:647] T 8c21a6762eaa4ba4ac1c91a42fb37849 P e6b60803d0764d228c863e2da65e3691: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:19.137737 2417 tablet_copy_client.cc:523] T 075d3240e24a4532bbb29a068c00c128 P db17f26a73f3447b92aa9a947e3a7057: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:19.138696 2422 tablet_copy_client.cc:523] T 8c21a6762eaa4ba4ac1c91a42fb37849 P e6b60803d0764d228c863e2da65e3691: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:19.139168 2417 tablet_bootstrap.cc:492] T 075d3240e24a4532bbb29a068c00c128 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:19.141109 2422 tablet_bootstrap.cc:492] T 8c21a6762eaa4ba4ac1c91a42fb37849 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:19.142575 32108 raft_consensus.cc:482] T fcf606426b8b44bcb660d6c2ed4af6e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:19.142663 32108 raft_consensus.cc:504] T fcf606426b8b44bcb660d6c2ed4af6e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:19.142846 2417 tablet_bootstrap.cc:492] T 075d3240e24a4532bbb29a068c00c128 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:19.143033 32108 leader_election.cc:288] T fcf606426b8b44bcb660d6c2ed4af6e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:19.143325 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf606426b8b44bcb660d6c2ed4af6e3" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:19.143406 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf606426b8b44bcb660d6c2ed4af6e3" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:19.143467 31757 raft_consensus.cc:2454] T fcf606426b8b44bcb660d6c2ed4af6e3 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:19.143522 31888 raft_consensus.cc:2454] T fcf606426b8b44bcb660d6c2ed4af6e3 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:19.143744 31429 leader_election.cc:302] T fcf606426b8b44bcb660d6c2ed4af6e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:19.143895 32108 raft_consensus.cc:2781] T fcf606426b8b44bcb660d6c2ed4af6e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:19.143946 32108 raft_consensus.cc:482] T fcf606426b8b44bcb660d6c2ed4af6e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:19.143981 32108 raft_consensus.cc:3037] T fcf606426b8b44bcb660d6c2ed4af6e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:19.144124 2417 tablet_bootstrap.cc:492] T 075d3240e24a4532bbb29a068c00c128 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap complete.
I20240502 20:46:19.144289 2417 ts_tablet_manager.cc:1367] T 075d3240e24a4532bbb29a068c00c128 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240502 20:46:19.144557 2315 tablet.cc:1722] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:19.144716 2417 raft_consensus.cc:348] T 075d3240e24a4532bbb29a068c00c128 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:19.144812 2417 raft_consensus.cc:729] T 075d3240e24a4532bbb29a068c00c128 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.144903 32108 raft_consensus.cc:504] T fcf606426b8b44bcb660d6c2ed4af6e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:19.144989 2317 tablet.cc:1722] T f107709bdafc4d24aca6f5c21cd265cf P db17f26a73f3447b92aa9a947e3a7057: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:19.145257 32108 leader_election.cc:288] T fcf606426b8b44bcb660d6c2ed4af6e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Requested vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:19.145612 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf606426b8b44bcb660d6c2ed4af6e3" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:19.145727 31888 raft_consensus.cc:3037] T fcf606426b8b44bcb660d6c2ed4af6e3 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:19.146634 31888 raft_consensus.cc:2454] T fcf606426b8b44bcb660d6c2ed4af6e3 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:19.146835 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf606426b8b44bcb660d6c2ed4af6e3" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:19.146958 31757 raft_consensus.cc:3037] T fcf606426b8b44bcb660d6c2ed4af6e3 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:19.147137 31429 leader_election.cc:302] T fcf606426b8b44bcb660d6c2ed4af6e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:19.147292 32108 raft_consensus.cc:2781] T fcf606426b8b44bcb660d6c2ed4af6e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:19.147420 32108 raft_consensus.cc:686] T fcf606426b8b44bcb660d6c2ed4af6e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Becoming Leader. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Running, Role: LEADER
I20240502 20:46:19.147548 32108 consensus_queue.cc:237] T fcf606426b8b44bcb660d6c2ed4af6e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:19.147917 31757 raft_consensus.cc:2454] T fcf606426b8b44bcb660d6c2ed4af6e3 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:19.144977 2417 consensus_queue.cc:260] T 075d3240e24a4532bbb29a068c00c128 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:19.148797 2417 ts_tablet_manager.cc:1398] T 075d3240e24a4532bbb29a068c00c128 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:19.149232 31908 tablet_copy_service.cc:342] P e6b60803d0764d228c863e2da65e3691: Request end of tablet copy session db17f26a73f3447b92aa9a947e3a7057-075d3240e24a4532bbb29a068c00c128 received from {username='slave'} at 127.22.45.68:38379
I20240502 20:46:19.149307 31908 tablet_copy_service.cc:433] P e6b60803d0764d228c863e2da65e3691: ending tablet copy session db17f26a73f3447b92aa9a947e3a7057-075d3240e24a4532bbb29a068c00c128 on tablet 075d3240e24a4532bbb29a068c00c128 with peer db17f26a73f3447b92aa9a947e3a7057
I20240502 20:46:19.150120 2316 tablet.cc:1722] T f107709bdafc4d24aca6f5c21cd265cf P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:19.152987 29034 catalog_manager.cc:5484] T fcf606426b8b44bcb660d6c2ed4af6e3 P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69). New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:19.154352 29036 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=98b6abad835643239e91d41d5a0fedec] alter complete (version 3)
I20240502 20:46:19.161489 2144 raft_consensus.cc:482] T 696d094d24fe48a3aeeb01e3b15a50e4 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:19.161607 2144 raft_consensus.cc:504] T 696d094d24fe48a3aeeb01e3b15a50e4 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:19.162911 2144 leader_election.cc:288] T 696d094d24fe48a3aeeb01e3b15a50e4 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:19.163190 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "696d094d24fe48a3aeeb01e3b15a50e4" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:19.163214 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "696d094d24fe48a3aeeb01e3b15a50e4" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:19.163323 31757 raft_consensus.cc:2454] T 696d094d24fe48a3aeeb01e3b15a50e4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:19.163357 29315 raft_consensus.cc:2454] T 696d094d24fe48a3aeeb01e3b15a50e4 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:19.163617 29117 leader_election.cc:302] T 696d094d24fe48a3aeeb01e3b15a50e4 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:19.164050 2144 raft_consensus.cc:2781] T 696d094d24fe48a3aeeb01e3b15a50e4 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:19.164110 2144 raft_consensus.cc:482] T 696d094d24fe48a3aeeb01e3b15a50e4 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:19.164142 2144 raft_consensus.cc:3037] T 696d094d24fe48a3aeeb01e3b15a50e4 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:19.165208 2144 raft_consensus.cc:504] T 696d094d24fe48a3aeeb01e3b15a50e4 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:19.165521 2144 leader_election.cc:288] T 696d094d24fe48a3aeeb01e3b15a50e4 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:19.165944 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "696d094d24fe48a3aeeb01e3b15a50e4" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:19.166052 31757 raft_consensus.cc:3037] T 696d094d24fe48a3aeeb01e3b15a50e4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:19.166043 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "696d094d24fe48a3aeeb01e3b15a50e4" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:19.166150 29315 raft_consensus.cc:3037] T 696d094d24fe48a3aeeb01e3b15a50e4 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:19.167094 31757 raft_consensus.cc:2454] T 696d094d24fe48a3aeeb01e3b15a50e4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:19.167178 29315 raft_consensus.cc:2454] T 696d094d24fe48a3aeeb01e3b15a50e4 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:19.167397 29117 leader_election.cc:302] T 696d094d24fe48a3aeeb01e3b15a50e4 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:19.167825 2144 raft_consensus.cc:2781] T 696d094d24fe48a3aeeb01e3b15a50e4 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:19.167897 2144 raft_consensus.cc:686] T 696d094d24fe48a3aeeb01e3b15a50e4 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:19.168021 2144 consensus_queue.cc:237] T 696d094d24fe48a3aeeb01e3b15a50e4 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:19.169919 29034 catalog_manager.cc:5484] T 696d094d24fe48a3aeeb01e3b15a50e4 P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:19.188647 2422 tablet_bootstrap.cc:492] T 8c21a6762eaa4ba4ac1c91a42fb37849 P e6b60803d0764d228c863e2da65e3691: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:19.194777 2422 tablet_bootstrap.cc:492] T 8c21a6762eaa4ba4ac1c91a42fb37849 P e6b60803d0764d228c863e2da65e3691: Bootstrap complete.
I20240502 20:46:19.195156 2422 ts_tablet_manager.cc:1367] T 8c21a6762eaa4ba4ac1c91a42fb37849 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.054s user 0.026s sys 0.000s
I20240502 20:46:19.195680 2422 raft_consensus.cc:348] T 8c21a6762eaa4ba4ac1c91a42fb37849 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:19.195801 2422 raft_consensus.cc:729] T 8c21a6762eaa4ba4ac1c91a42fb37849 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.202715 2422 consensus_queue.cc:260] T 8c21a6762eaa4ba4ac1c91a42fb37849 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:19.203090 2422 ts_tablet_manager.cc:1398] T 8c21a6762eaa4ba4ac1c91a42fb37849 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20240502 20:46:19.204380 29202 tablet_copy_service.cc:342] P 7796ea3fba294d17a34386873969431d: Request end of tablet copy session e6b60803d0764d228c863e2da65e3691-8c21a6762eaa4ba4ac1c91a42fb37849 received from {username='slave'} at 127.22.45.72:55003
I20240502 20:46:19.204481 29202 tablet_copy_service.cc:433] P 7796ea3fba294d17a34386873969431d: ending tablet copy session e6b60803d0764d228c863e2da65e3691-8c21a6762eaa4ba4ac1c91a42fb37849 on tablet 8c21a6762eaa4ba4ac1c91a42fb37849 with peer e6b60803d0764d228c863e2da65e3691
I20240502 20:46:19.230901 31475 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:19.231323 29162 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:19.231844 31602 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:19.236845 29424 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:19.239351 29293 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:19.241298 29557 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:19.249168 31736 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:19.253369 31868 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:19.258651 29033 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:19.258826 29033 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:19.261021 29033 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet f107709bdafc4d24aca6f5c21cd265cf
I20240502 20:46:19.261510 31475 tablet_service.cc:1514] Processing DeleteTablet for tablet f107709bdafc4d24aca6f5c21cd265cf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:19 UTC) from {username='slave'} at 127.0.0.1:40276
I20240502 20:46:19.261840 2475 tablet_replica.cc:330] T f107709bdafc4d24aca6f5c21cd265cf P 2a9c1af8cf814030bde56b67ca5c2fbe: stopping tablet replica
I20240502 20:46:19.261906 29293 tablet_service.cc:1514] Processing DeleteTablet for tablet f107709bdafc4d24aca6f5c21cd265cf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:19 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:19.261979 2475 raft_consensus.cc:2229] T f107709bdafc4d24aca6f5c21cd265cf P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:19.262086 2475 raft_consensus.cc:2258] T f107709bdafc4d24aca6f5c21cd265cf P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:19.262668 2475 ts_tablet_manager.cc:1875] T f107709bdafc4d24aca6f5c21cd265cf P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:19.264957 2475 ts_tablet_manager.cc:1888] T f107709bdafc4d24aca6f5c21cd265cf P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:19.265084 2475 log.cc:1196] T f107709bdafc4d24aca6f5c21cd265cf P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/wals/f107709bdafc4d24aca6f5c21cd265cf
I20240502 20:46:19.266010 2475 ts_tablet_manager.cc:1909] T f107709bdafc4d24aca6f5c21cd265cf P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting consensus metadata
I20240502 20:46:19.266741 29028 catalog_manager.cc:4830] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): tablet f107709bdafc4d24aca6f5c21cd265cf (table rebalancer_dynamic_table_new_name [id=98b6abad835643239e91d41d5a0fedec]) successfully deleted
I20240502 20:46:19.267129 29557 tablet_service.cc:1514] Processing DeleteTablet for tablet f107709bdafc4d24aca6f5c21cd265cf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:19 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:19.267454 2477 tablet_replica.cc:330] T f107709bdafc4d24aca6f5c21cd265cf P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:19.267639 2477 raft_consensus.cc:2229] T f107709bdafc4d24aca6f5c21cd265cf P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:19.267787 2477 raft_consensus.cc:2258] T f107709bdafc4d24aca6f5c21cd265cf P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:19.268406 2477 ts_tablet_manager.cc:1875] T f107709bdafc4d24aca6f5c21cd265cf P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:19.270411 2477 ts_tablet_manager.cc:1888] T f107709bdafc4d24aca6f5c21cd265cf P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:19.270502 2477 log.cc:1196] T f107709bdafc4d24aca6f5c21cd265cf P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/f107709bdafc4d24aca6f5c21cd265cf
I20240502 20:46:19.270805 2477 ts_tablet_manager.cc:1909] T f107709bdafc4d24aca6f5c21cd265cf P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:19.271407 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet f107709bdafc4d24aca6f5c21cd265cf (table rebalancer_dynamic_table_new_name [id=98b6abad835643239e91d41d5a0fedec]) successfully deleted
I20240502 20:46:19.272105 2476 tablet_replica.cc:330] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:19.272253 2476 raft_consensus.cc:2229] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:19.272471 2476 raft_consensus.cc:2258] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:19.274176 2476 ts_tablet_manager.cc:1875] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:19.276382 2476 ts_tablet_manager.cc:1888] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:19.276494 2476 log.cc:1196] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/f107709bdafc4d24aca6f5c21cd265cf
I20240502 20:46:19.276851 2476 ts_tablet_manager.cc:1909] T f107709bdafc4d24aca6f5c21cd265cf P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:19.277475 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet f107709bdafc4d24aca6f5c21cd265cf (table rebalancer_dynamic_table_new_name [id=98b6abad835643239e91d41d5a0fedec]) successfully deleted
I20240502 20:46:19.329227 2495 ts_tablet_manager.cc:902] T 6b675480162c4a9799db51ea4b6504a5 P 09258ac095664fa99caf0707262cde13: Initiating tablet copy from peer e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:19.329792 2495 tablet_copy_client.cc:308] T 6b675480162c4a9799db51ea4b6504a5 P 09258ac095664fa99caf0707262cde13: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.72:35207
I20240502 20:46:19.330236 31908 tablet_copy_service.cc:140] P e6b60803d0764d228c863e2da65e3691: Received BeginTabletCopySession request for tablet 6b675480162c4a9799db51ea4b6504a5 from peer 09258ac095664fa99caf0707262cde13 ({username='slave'} at 127.22.45.66:55365)
I20240502 20:46:19.330351 31908 tablet_copy_service.cc:161] P e6b60803d0764d228c863e2da65e3691: Beginning new tablet copy session on tablet 6b675480162c4a9799db51ea4b6504a5 from peer 09258ac095664fa99caf0707262cde13 at {username='slave'} at 127.22.45.66:55365: session id = 09258ac095664fa99caf0707262cde13-6b675480162c4a9799db51ea4b6504a5
I20240502 20:46:19.331224 31908 tablet_copy_source_session.cc:205] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:19.332479 2495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b675480162c4a9799db51ea4b6504a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:19.334441 2495 tablet_copy_client.cc:783] T 6b675480162c4a9799db51ea4b6504a5 P 09258ac095664fa99caf0707262cde13: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:19.334610 2495 tablet_copy_client.cc:647] T 6b675480162c4a9799db51ea4b6504a5 P 09258ac095664fa99caf0707262cde13: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:19.335546 2386 ts_tablet_manager.cc:902] T d4a2c7b197f948a0b236d5e05e2749b9 P 315915efcf2b413ca7f9d874338e1b2d: Initiating tablet copy from peer 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:19.335956 2386 tablet_copy_client.cc:308] T d4a2c7b197f948a0b236d5e05e2749b9 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.66:40945
I20240502 20:46:19.336462 29335 tablet_copy_service.cc:140] P 09258ac095664fa99caf0707262cde13: Received BeginTabletCopySession request for tablet d4a2c7b197f948a0b236d5e05e2749b9 from peer 315915efcf2b413ca7f9d874338e1b2d ({username='slave'} at 127.22.45.70:37603)
I20240502 20:46:19.336553 29335 tablet_copy_service.cc:161] P 09258ac095664fa99caf0707262cde13: Beginning new tablet copy session on tablet d4a2c7b197f948a0b236d5e05e2749b9 from peer 315915efcf2b413ca7f9d874338e1b2d at {username='slave'} at 127.22.45.70:37603: session id = 315915efcf2b413ca7f9d874338e1b2d-d4a2c7b197f948a0b236d5e05e2749b9
I20240502 20:46:19.336726 2495 tablet_copy_client.cc:523] T 6b675480162c4a9799db51ea4b6504a5 P 09258ac095664fa99caf0707262cde13: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:19.337329 29335 tablet_copy_source_session.cc:205] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:19.338083 2386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4a2c7b197f948a0b236d5e05e2749b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:19.340180 2386 tablet_copy_client.cc:783] T d4a2c7b197f948a0b236d5e05e2749b9 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:19.340319 2386 tablet_copy_client.cc:647] T d4a2c7b197f948a0b236d5e05e2749b9 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:19.340873 2495 tablet_bootstrap.cc:492] T 6b675480162c4a9799db51ea4b6504a5 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:19.341344 2386 tablet_copy_client.cc:523] T d4a2c7b197f948a0b236d5e05e2749b9 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:19.342301 2386 tablet_bootstrap.cc:492] T d4a2c7b197f948a0b236d5e05e2749b9 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:19.344760 2495 tablet_bootstrap.cc:492] T 6b675480162c4a9799db51ea4b6504a5 P 09258ac095664fa99caf0707262cde13: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:19.344983 2495 tablet_bootstrap.cc:492] T 6b675480162c4a9799db51ea4b6504a5 P 09258ac095664fa99caf0707262cde13: Bootstrap complete.
I20240502 20:46:19.345134 2495 ts_tablet_manager.cc:1367] T 6b675480162c4a9799db51ea4b6504a5 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 20:46:19.345562 2495 raft_consensus.cc:348] T 6b675480162c4a9799db51ea4b6504a5 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:19.345685 2495 raft_consensus.cc:729] T 6b675480162c4a9799db51ea4b6504a5 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.345837 2495 consensus_queue.cc:260] T 6b675480162c4a9799db51ea4b6504a5 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:19.346845 2495 ts_tablet_manager.cc:1398] T 6b675480162c4a9799db51ea4b6504a5 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240502 20:46:19.347270 31908 tablet_copy_service.cc:342] P e6b60803d0764d228c863e2da65e3691: Request end of tablet copy session 09258ac095664fa99caf0707262cde13-6b675480162c4a9799db51ea4b6504a5 received from {username='slave'} at 127.22.45.66:55365
I20240502 20:46:19.347350 31908 tablet_copy_service.cc:433] P e6b60803d0764d228c863e2da65e3691: ending tablet copy session 09258ac095664fa99caf0707262cde13-6b675480162c4a9799db51ea4b6504a5 on tablet 6b675480162c4a9799db51ea4b6504a5 with peer 09258ac095664fa99caf0707262cde13
I20240502 20:46:19.363642 29035 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:19.370867 2386 tablet_bootstrap.cc:492] T d4a2c7b197f948a0b236d5e05e2749b9 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:19.371196 2386 tablet_bootstrap.cc:492] T d4a2c7b197f948a0b236d5e05e2749b9 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap complete.
I20240502 20:46:19.371383 2386 ts_tablet_manager.cc:1367] T d4a2c7b197f948a0b236d5e05e2749b9 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.029s user 0.025s sys 0.000s
I20240502 20:46:19.371845 2386 raft_consensus.cc:348] T d4a2c7b197f948a0b236d5e05e2749b9 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:19.371973 2386 raft_consensus.cc:729] T d4a2c7b197f948a0b236d5e05e2749b9 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.372116 2386 consensus_queue.cc:260] T d4a2c7b197f948a0b236d5e05e2749b9 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:19.372560 31736 tablet_service.cc:1467] Processing CreateTablet for tablet ca4d881c6ab943b99d43dd775b9d7281 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=1422daa42e834e10bc5866663d406d10]), partition=
I20240502 20:46:19.372808 31736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca4d881c6ab943b99d43dd775b9d7281. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:19.373035 2386 ts_tablet_manager.cc:1398] T d4a2c7b197f948a0b236d5e05e2749b9 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:19.374498 29335 tablet_copy_service.cc:342] P 09258ac095664fa99caf0707262cde13: Request end of tablet copy session 315915efcf2b413ca7f9d874338e1b2d-d4a2c7b197f948a0b236d5e05e2749b9 received from {username='slave'} at 127.22.45.70:37603
I20240502 20:46:19.374846 29335 tablet_copy_service.cc:433] P 09258ac095664fa99caf0707262cde13: ending tablet copy session 315915efcf2b413ca7f9d874338e1b2d-d4a2c7b197f948a0b236d5e05e2749b9 on tablet d4a2c7b197f948a0b236d5e05e2749b9 with peer 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:19.375885 31602 tablet_service.cc:1467] Processing CreateTablet for tablet ca4d881c6ab943b99d43dd775b9d7281 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=1422daa42e834e10bc5866663d406d10]), partition=
I20240502 20:46:19.376132 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca4d881c6ab943b99d43dd775b9d7281. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:19.378527 31868 tablet_service.cc:1467] Processing CreateTablet for tablet ca4d881c6ab943b99d43dd775b9d7281 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=1422daa42e834e10bc5866663d406d10]), partition=
I20240502 20:46:19.378770 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca4d881c6ab943b99d43dd775b9d7281. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:19.381011 2377 tablet_bootstrap.cc:492] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:19.381877 2377 tablet_bootstrap.cc:654] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:19.382970 2377 tablet_bootstrap.cc:492] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:19.383054 2377 ts_tablet_manager.cc:1367] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:19.383478 2377 raft_consensus.cc:348] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:19.383576 2377 raft_consensus.cc:374] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:19.383601 2377 raft_consensus.cc:729] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.383716 2377 consensus_queue.cc:260] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:19.383965 2377 ts_tablet_manager.cc:1398] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:19.385066 2531 ts_tablet_manager.cc:902] T 58d4026c208f45d8870a2eaa1c97efab P e7ab6b9053664ee5aa71c7978e26fe6a: Initiating tablet copy from peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:19.385685 2531 tablet_copy_client.cc:308] T 58d4026c208f45d8870a2eaa1c97efab P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.70:46751
I20240502 20:46:19.385993 2532 tablet_bootstrap.cc:492] T ca4d881c6ab943b99d43dd775b9d7281 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:19.385707 2374 tablet_bootstrap.cc:492] T ca4d881c6ab943b99d43dd775b9d7281 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:19.388960 31646 tablet_copy_service.cc:140] P 315915efcf2b413ca7f9d874338e1b2d: Received BeginTabletCopySession request for tablet 58d4026c208f45d8870a2eaa1c97efab from peer e7ab6b9053664ee5aa71c7978e26fe6a ({username='slave'} at 127.22.45.71:48751)
I20240502 20:46:19.389068 31646 tablet_copy_service.cc:161] P 315915efcf2b413ca7f9d874338e1b2d: Beginning new tablet copy session on tablet 58d4026c208f45d8870a2eaa1c97efab from peer e7ab6b9053664ee5aa71c7978e26fe6a at {username='slave'} at 127.22.45.71:48751: session id = e7ab6b9053664ee5aa71c7978e26fe6a-58d4026c208f45d8870a2eaa1c97efab
I20240502 20:46:19.389446 2374 tablet_bootstrap.cc:654] T ca4d881c6ab943b99d43dd775b9d7281 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:19.389968 31646 tablet_copy_source_session.cc:205] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:19.390867 2531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58d4026c208f45d8870a2eaa1c97efab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:19.391896 2386 ts_tablet_manager.cc:902] T 4a91eafc5ecf45b09f60d712884b7c79 P 315915efcf2b413ca7f9d874338e1b2d: Initiating tablet copy from peer dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:19.392220 2532 tablet_bootstrap.cc:654] T ca4d881c6ab943b99d43dd775b9d7281 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:19.392375 2386 tablet_copy_client.cc:308] T 4a91eafc5ecf45b09f60d712884b7c79 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.67:44095
I20240502 20:46:19.393153 29465 tablet_copy_service.cc:140] P dbcdc6ac635b44b185c3eba63f66e9d6: Received BeginTabletCopySession request for tablet 4a91eafc5ecf45b09f60d712884b7c79 from peer 315915efcf2b413ca7f9d874338e1b2d ({username='slave'} at 127.22.45.70:49775)
I20240502 20:46:19.393270 29465 tablet_copy_service.cc:161] P dbcdc6ac635b44b185c3eba63f66e9d6: Beginning new tablet copy session on tablet 4a91eafc5ecf45b09f60d712884b7c79 from peer 315915efcf2b413ca7f9d874338e1b2d at {username='slave'} at 127.22.45.70:49775: session id = 315915efcf2b413ca7f9d874338e1b2d-4a91eafc5ecf45b09f60d712884b7c79
I20240502 20:46:19.394147 29465 tablet_copy_source_session.cc:205] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:19.394538 2422 ts_tablet_manager.cc:902] T a2bf196bcab440c48aa4ea893997573b P e6b60803d0764d228c863e2da65e3691: Initiating tablet copy from peer dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:19.395006 2422 tablet_copy_client.cc:308] T a2bf196bcab440c48aa4ea893997573b P e6b60803d0764d228c863e2da65e3691: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.67:44095
I20240502 20:46:19.395424 2386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a91eafc5ecf45b09f60d712884b7c79. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:19.395538 29465 tablet_copy_service.cc:140] P dbcdc6ac635b44b185c3eba63f66e9d6: Received BeginTabletCopySession request for tablet a2bf196bcab440c48aa4ea893997573b from peer e6b60803d0764d228c863e2da65e3691 ({username='slave'} at 127.22.45.72:44455)
I20240502 20:46:19.395632 29465 tablet_copy_service.cc:161] P dbcdc6ac635b44b185c3eba63f66e9d6: Beginning new tablet copy session on tablet a2bf196bcab440c48aa4ea893997573b from peer e6b60803d0764d228c863e2da65e3691 at {username='slave'} at 127.22.45.72:44455: session id = e6b60803d0764d228c863e2da65e3691-a2bf196bcab440c48aa4ea893997573b
I20240502 20:46:19.396458 2374 tablet_bootstrap.cc:492] T ca4d881c6ab943b99d43dd775b9d7281 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:19.396481 29465 tablet_copy_source_session.cc:205] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:19.396553 2374 ts_tablet_manager.cc:1367] T ca4d881c6ab943b99d43dd775b9d7281 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 20:46:19.396991 2374 raft_consensus.cc:348] T ca4d881c6ab943b99d43dd775b9d7281 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:19.397104 2374 raft_consensus.cc:374] T ca4d881c6ab943b99d43dd775b9d7281 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:19.397136 2374 raft_consensus.cc:729] T ca4d881c6ab943b99d43dd775b9d7281 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.397245 2374 consensus_queue.cc:260] T ca4d881c6ab943b99d43dd775b9d7281 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:19.397369 2422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2bf196bcab440c48aa4ea893997573b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:19.397454 2386 tablet_copy_client.cc:783] T 4a91eafc5ecf45b09f60d712884b7c79 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:19.397509 2374 ts_tablet_manager.cc:1398] T ca4d881c6ab943b99d43dd775b9d7281 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:19.397572 2386 tablet_copy_client.cc:647] T 4a91eafc5ecf45b09f60d712884b7c79 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:19.397891 2494 raft_consensus.cc:482] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:19.397974 2494 raft_consensus.cc:504] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:19.398347 2494 leader_election.cc:288] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:19.400034 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca4d881c6ab943b99d43dd775b9d7281" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:19.400661 2422 tablet_copy_client.cc:783] T a2bf196bcab440c48aa4ea893997573b P e6b60803d0764d228c863e2da65e3691: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:19.400796 2422 tablet_copy_client.cc:647] T a2bf196bcab440c48aa4ea893997573b P e6b60803d0764d228c863e2da65e3691: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:19.401150 2531 tablet_copy_client.cc:783] T 58d4026c208f45d8870a2eaa1c97efab P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:19.401294 2531 tablet_copy_client.cc:647] T 58d4026c208f45d8870a2eaa1c97efab P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:19.401577 2386 tablet_copy_client.cc:523] T 4a91eafc5ecf45b09f60d712884b7c79 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:19.402382 2531 tablet_copy_client.cc:523] T 58d4026c208f45d8870a2eaa1c97efab P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20240502 20:46:19.402971 31822 leader_election.cc:341] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Illegal state: must be running to vote when last-logged opid is not known
I20240502 20:46:19.403033 2386 tablet_bootstrap.cc:492] T 4a91eafc5ecf45b09f60d712884b7c79 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:19.404867 2531 tablet_bootstrap.cc:492] T 58d4026c208f45d8870a2eaa1c97efab P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:19.406795 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca4d881c6ab943b99d43dd775b9d7281" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:19.406937 31757 raft_consensus.cc:2454] T ca4d881c6ab943b99d43dd775b9d7281 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:19.407076 2532 tablet_bootstrap.cc:492] T ca4d881c6ab943b99d43dd775b9d7281 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:19.407189 2532 ts_tablet_manager.cc:1367] T ca4d881c6ab943b99d43dd775b9d7281 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20240502 20:46:19.407256 31821 leader_election.cc:302] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e6b60803d0764d228c863e2da65e3691, e7ab6b9053664ee5aa71c7978e26fe6a; no voters: 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:19.407493 2494 raft_consensus.cc:2781] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:19.407550 2494 raft_consensus.cc:482] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:19.407590 2494 raft_consensus.cc:3037] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:19.407778 2532 raft_consensus.cc:348] T ca4d881c6ab943b99d43dd775b9d7281 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:19.407907 2532 raft_consensus.cc:374] T ca4d881c6ab943b99d43dd775b9d7281 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:19.407954 2532 raft_consensus.cc:729] T ca4d881c6ab943b99d43dd775b9d7281 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.408105 2532 consensus_queue.cc:260] T ca4d881c6ab943b99d43dd775b9d7281 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:19.408535 2494 raft_consensus.cc:504] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:19.408900 2494 leader_election.cc:288] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Requested vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:19.409040 2532 ts_tablet_manager.cc:1398] T ca4d881c6ab943b99d43dd775b9d7281 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:19.409929 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca4d881c6ab943b99d43dd775b9d7281" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:19.409930 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca4d881c6ab943b99d43dd775b9d7281" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:19.410034 31625 raft_consensus.cc:3037] T ca4d881c6ab943b99d43dd775b9d7281 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:19.410040 31757 raft_consensus.cc:3037] T ca4d881c6ab943b99d43dd775b9d7281 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:19.410274 2422 tablet_copy_client.cc:523] T a2bf196bcab440c48aa4ea893997573b P e6b60803d0764d228c863e2da65e3691: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:19.410988 31625 raft_consensus.cc:2454] T ca4d881c6ab943b99d43dd775b9d7281 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:19.411336 31822 leader_election.cc:302] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:19.411558 2422 tablet_bootstrap.cc:492] T a2bf196bcab440c48aa4ea893997573b P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:19.411587 2494 raft_consensus.cc:2781] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:19.411661 2494 raft_consensus.cc:686] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Becoming Leader. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Running, Role: LEADER
I20240502 20:46:19.411782 2494 consensus_queue.cc:237] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:19.414669 31757 raft_consensus.cc:2454] T ca4d881c6ab943b99d43dd775b9d7281 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:19.416066 29037 catalog_manager.cc:5484] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6b60803d0764d228c863e2da65e3691 (127.22.45.72). New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:19.427356 2386 tablet_bootstrap.cc:492] T 4a91eafc5ecf45b09f60d712884b7c79 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:19.427685 2386 tablet_bootstrap.cc:492] T 4a91eafc5ecf45b09f60d712884b7c79 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap complete.
I20240502 20:46:19.427865 2386 ts_tablet_manager.cc:1367] T 4a91eafc5ecf45b09f60d712884b7c79 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.025s user 0.023s sys 0.000s
I20240502 20:46:19.428361 2386 raft_consensus.cc:348] T 4a91eafc5ecf45b09f60d712884b7c79 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:19.428493 2386 raft_consensus.cc:729] T 4a91eafc5ecf45b09f60d712884b7c79 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.428889 2386 consensus_queue.cc:260] T 4a91eafc5ecf45b09f60d712884b7c79 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:19.429260 2386 ts_tablet_manager.cc:1398] T 4a91eafc5ecf45b09f60d712884b7c79 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 20:46:19.429805 29465 tablet_copy_service.cc:342] P dbcdc6ac635b44b185c3eba63f66e9d6: Request end of tablet copy session 315915efcf2b413ca7f9d874338e1b2d-4a91eafc5ecf45b09f60d712884b7c79 received from {username='slave'} at 127.22.45.70:49775
I20240502 20:46:19.429891 29465 tablet_copy_service.cc:433] P dbcdc6ac635b44b185c3eba63f66e9d6: ending tablet copy session 315915efcf2b413ca7f9d874338e1b2d-4a91eafc5ecf45b09f60d712884b7c79 on tablet 4a91eafc5ecf45b09f60d712884b7c79 with peer 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:19.431437 2531 tablet_bootstrap.cc:492] T 58d4026c208f45d8870a2eaa1c97efab P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:19.431699 2531 tablet_bootstrap.cc:492] T 58d4026c208f45d8870a2eaa1c97efab P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap complete.
I20240502 20:46:19.431874 2531 ts_tablet_manager.cc:1367] T 58d4026c208f45d8870a2eaa1c97efab P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.002s
I20240502 20:46:19.432397 2531 raft_consensus.cc:348] T 58d4026c208f45d8870a2eaa1c97efab P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:19.432507 2531 raft_consensus.cc:729] T 58d4026c208f45d8870a2eaa1c97efab P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.432619 2531 consensus_queue.cc:260] T 58d4026c208f45d8870a2eaa1c97efab P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:19.432832 2531 ts_tablet_manager.cc:1398] T 58d4026c208f45d8870a2eaa1c97efab P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:19.433840 31646 tablet_copy_service.cc:342] P 315915efcf2b413ca7f9d874338e1b2d: Request end of tablet copy session e7ab6b9053664ee5aa71c7978e26fe6a-58d4026c208f45d8870a2eaa1c97efab received from {username='slave'} at 127.22.45.71:48751
I20240502 20:46:19.433910 31646 tablet_copy_service.cc:433] P 315915efcf2b413ca7f9d874338e1b2d: ending tablet copy session e7ab6b9053664ee5aa71c7978e26fe6a-58d4026c208f45d8870a2eaa1c97efab on tablet 58d4026c208f45d8870a2eaa1c97efab with peer e7ab6b9053664ee5aa71c7978e26fe6a
I20240502 20:46:19.443408 2422 tablet_bootstrap.cc:492] T a2bf196bcab440c48aa4ea893997573b P e6b60803d0764d228c863e2da65e3691: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:19.443964 2422 tablet_bootstrap.cc:492] T a2bf196bcab440c48aa4ea893997573b P e6b60803d0764d228c863e2da65e3691: Bootstrap complete.
I20240502 20:46:19.444178 2422 ts_tablet_manager.cc:1367] T a2bf196bcab440c48aa4ea893997573b P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.033s user 0.024s sys 0.002s
I20240502 20:46:19.444720 2422 raft_consensus.cc:348] T a2bf196bcab440c48aa4ea893997573b P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:19.444922 2422 raft_consensus.cc:729] T a2bf196bcab440c48aa4ea893997573b P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.445150 2422 consensus_queue.cc:260] T a2bf196bcab440c48aa4ea893997573b P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:19.445535 2422 ts_tablet_manager.cc:1398] T a2bf196bcab440c48aa4ea893997573b P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:19.446146 29465 tablet_copy_service.cc:342] P dbcdc6ac635b44b185c3eba63f66e9d6: Request end of tablet copy session e6b60803d0764d228c863e2da65e3691-a2bf196bcab440c48aa4ea893997573b received from {username='slave'} at 127.22.45.72:44455
I20240502 20:46:19.446238 29465 tablet_copy_service.cc:433] P dbcdc6ac635b44b185c3eba63f66e9d6: ending tablet copy session e6b60803d0764d228c863e2da65e3691-a2bf196bcab440c48aa4ea893997573b on tablet a2bf196bcab440c48aa4ea893997573b with peer e6b60803d0764d228c863e2da65e3691
I20240502 20:46:19.461575 29162 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:19.465169 29182 raft_consensus.cc:1203] T f14cf6deefec46c7adb598f6d7cfd649 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 20:46:19.467655 31602 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:19.469023 29293 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:19.474635 31475 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:19.474953 29557 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:19.483816 31736 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:19.490170 31868 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:19.497119 29424 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:19.527340 29038 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:19.531795 31625 raft_consensus.cc:1261] T ca4d881c6ab943b99d43dd775b9d7281 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:19.531971 31757 raft_consensus.cc:1261] T ca4d881c6ab943b99d43dd775b9d7281 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:19.532287 2494 consensus_queue.cc:1035] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:19.532701 2494 consensus_queue.cc:1035] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:19.536448 29035 catalog_manager.cc:5576] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751) does not have the latest schema for tablet ca4d881c6ab943b99d43dd775b9d7281 (table rebalancer_dynamic_table [id=1422daa42e834e10bc5866663d406d10]). Expected version 1 got 0
I20240502 20:46:19.537283 2584 tablet.cc:1722] T ca4d881c6ab943b99d43dd775b9d7281 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:19.537863 2584 tablet.cc:1620] T ca4d881c6ab943b99d43dd775b9d7281 P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:19.538295 29035 catalog_manager.cc:5576] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751) does not have the latest schema for tablet ca4d881c6ab943b99d43dd775b9d7281 (table rebalancer_dynamic_table [id=1422daa42e834e10bc5866663d406d10]). Expected version 1 got 0
I20240502 20:46:19.539592 29035 catalog_manager.cc:5576] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787) does not have the latest schema for tablet ca4d881c6ab943b99d43dd775b9d7281 (table rebalancer_dynamic_table [id=1422daa42e834e10bc5866663d406d10]). Expected version 1 got 0
I20240502 20:46:19.540635 2583 tablet.cc:1722] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:19.540935 2583 tablet.cc:1620] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:19.542946 2184 consensus_queue.cc:1035] T 1ce967c38f6f48c7a14b837335d1deca P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:19.543287 29042 catalog_manager.cc:5576] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787) does not have the latest schema for tablet ca4d881c6ab943b99d43dd775b9d7281 (table rebalancer_dynamic_table [id=1422daa42e834e10bc5866663d406d10]). Expected version 1 got 0
I20240502 20:46:19.545270 29035 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=1422daa42e834e10bc5866663d406d10] alter complete (version 1)
I20240502 20:46:19.548875 2590 tablet.cc:1722] T ca4d881c6ab943b99d43dd775b9d7281 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:19.549338 2590 tablet.cc:1620] T ca4d881c6ab943b99d43dd775b9d7281 P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:19.549995 692 consensus_queue.cc:1035] T 1ce967c38f6f48c7a14b837335d1deca P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:19.583348 31888 raft_consensus.cc:1203] T 8c21a6762eaa4ba4ac1c91a42fb37849 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:19.585757 29577 raft_consensus.cc:1203] T 075d3240e24a4532bbb29a068c00c128 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 20:46:19.592479 2434 consensus_queue.cc:1035] T fcf606426b8b44bcb660d6c2ed4af6e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:19.596525 2434 consensus_queue.cc:1035] T fcf606426b8b44bcb660d6c2ed4af6e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:19.607228 1776 consensus_queue.cc:1035] T 696d094d24fe48a3aeeb01e3b15a50e4 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:19.615355 31625 raft_consensus.cc:1203] T fbd2bd8e142741ba864e2f99610b39d6 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:19.621914 31495 raft_consensus.cc:1203] T 689b7368353a430dbffa1cf44f9b5949 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:19.623301 2144 consensus_queue.cc:1035] T 696d094d24fe48a3aeeb01e3b15a50e4 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:19.646687 29293 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:19.649224 29033 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:19.653764 29162 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:19.657025 2583 tablet.cc:1722] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:19.657366 2583 tablet.cc:1620] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:19.657555 2590 tablet.cc:1722] T ca4d881c6ab943b99d43dd775b9d7281 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:19.657835 2590 tablet.cc:1620] T ca4d881c6ab943b99d43dd775b9d7281 P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:19.658010 2584 tablet.cc:1722] T ca4d881c6ab943b99d43dd775b9d7281 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:19.658278 2584 tablet.cc:1620] T ca4d881c6ab943b99d43dd775b9d7281 P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:19.661971 31602 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 20:46:19.663748 29557 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:19.664136 29039 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=1422daa42e834e10bc5866663d406d10] alter complete (version 2)
I20240502 20:46:19.665545 29424 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:19.688087 31736 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:19.690081 31868 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:19.695459 31475 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:19.708935 29033 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_20"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:19.710651 29577 raft_consensus.cc:1203] T 03db15f1ebfb4e4ea271d6bb5c76110d P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 20:46:19.717118 31602 tablet_service.cc:1467] Processing CreateTablet for tablet f5bdf4f57d2c4c828b94c7cc2e3c2847 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=15bc33de19bd4776861b116a13509551]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:19.717453 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5bdf4f57d2c4c828b94c7cc2e3c2847. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:19.717911 31736 tablet_service.cc:1467] Processing CreateTablet for tablet f5bdf4f57d2c4c828b94c7cc2e3c2847 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=15bc33de19bd4776861b116a13509551]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:19.718196 31736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5bdf4f57d2c4c828b94c7cc2e3c2847. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:19.718176 31475 tablet_service.cc:1467] Processing CreateTablet for tablet 8d5cca35ba4b47748235d1463ec161d6 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=15bc33de19bd4776861b116a13509551]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:19.718425 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d5cca35ba4b47748235d1463ec161d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:19.719688 2532 tablet_bootstrap.cc:492] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:19.720394 31735 tablet_service.cc:1467] Processing CreateTablet for tablet 8d5cca35ba4b47748235d1463ec161d6 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=15bc33de19bd4776861b116a13509551]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:19.720614 31735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d5cca35ba4b47748235d1463ec161d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:19.720671 2532 tablet_bootstrap.cc:654] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:19.720777 31606 tablet_service.cc:1467] Processing CreateTablet for tablet 1b2fc2ad5cd24a128562714712e89d89 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=15bc33de19bd4776861b116a13509551]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:19.720886 29163 tablet_service.cc:1467] Processing CreateTablet for tablet 1b2fc2ad5cd24a128562714712e89d89 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=15bc33de19bd4776861b116a13509551]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:19.720993 31606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b2fc2ad5cd24a128562714712e89d89. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:19.721129 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b2fc2ad5cd24a128562714712e89d89. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:19.722365 29162 tablet_service.cc:1467] Processing CreateTablet for tablet f5bdf4f57d2c4c828b94c7cc2e3c2847 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=15bc33de19bd4776861b116a13509551]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:19.722587 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5bdf4f57d2c4c828b94c7cc2e3c2847. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:19.723184 31602 tablet_service.cc:1467] Processing CreateTablet for tablet 8d5cca35ba4b47748235d1463ec161d6 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=15bc33de19bd4776861b116a13509551]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:19.723392 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d5cca35ba4b47748235d1463ec161d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:19.724200 2647 tablet_bootstrap.cc:492] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:19.724248 2374 tablet_bootstrap.cc:492] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:19.725042 2647 tablet_bootstrap.cc:654] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:19.725059 2374 tablet_bootstrap.cc:654] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:19.717931 31737 tablet_service.cc:1467] Processing CreateTablet for tablet 1b2fc2ad5cd24a128562714712e89d89 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=15bc33de19bd4776861b116a13509551]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:19.726181 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b2fc2ad5cd24a128562714712e89d89. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:19.727910 2645 tablet_bootstrap.cc:492] T 8d5cca35ba4b47748235d1463ec161d6 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:19.728825 2645 tablet_bootstrap.cc:654] T 8d5cca35ba4b47748235d1463ec161d6 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:19.729918 2374 tablet_bootstrap.cc:492] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:19.730003 2374 ts_tablet_manager.cc:1367] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 20:46:19.730396 2374 raft_consensus.cc:348] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:19.730522 2374 raft_consensus.cc:374] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:19.730568 2374 raft_consensus.cc:729] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.730674 2374 consensus_queue.cc:260] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:19.730940 2374 ts_tablet_manager.cc:1398] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:19.731165 2374 tablet_bootstrap.cc:492] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:19.732169 2374 tablet_bootstrap.cc:654] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:19.736744 2645 tablet_bootstrap.cc:492] T 8d5cca35ba4b47748235d1463ec161d6 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:19.737126 2645 ts_tablet_manager.cc:1367] T 8d5cca35ba4b47748235d1463ec161d6 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 20:46:19.737668 2645 raft_consensus.cc:348] T 8d5cca35ba4b47748235d1463ec161d6 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:19.737854 2645 raft_consensus.cc:374] T 8d5cca35ba4b47748235d1463ec161d6 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:19.737941 2645 raft_consensus.cc:729] T 8d5cca35ba4b47748235d1463ec161d6 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.738142 2645 consensus_queue.cc:260] T 8d5cca35ba4b47748235d1463ec161d6 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:19.739002 2645 ts_tablet_manager.cc:1398] T 8d5cca35ba4b47748235d1463ec161d6 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:19.739928 2647 tablet_bootstrap.cc:492] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:19.740053 2374 tablet_bootstrap.cc:492] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:19.740088 2647 ts_tablet_manager.cc:1367] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240502 20:46:19.740324 2374 ts_tablet_manager.cc:1367] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 20:46:19.740832 2374 raft_consensus.cc:348] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:19.741118 2374 raft_consensus.cc:374] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:19.741204 2374 raft_consensus.cc:729] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.741391 2374 consensus_queue.cc:260] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:19.741724 2374 ts_tablet_manager.cc:1398] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:19.742267 2374 tablet_bootstrap.cc:492] T 1b2fc2ad5cd24a128562714712e89d89 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:19.743502 2532 tablet_bootstrap.cc:492] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:19.743602 2532 ts_tablet_manager.cc:1367] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.000s
I20240502 20:46:19.743891 2374 tablet_bootstrap.cc:654] T 1b2fc2ad5cd24a128562714712e89d89 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:19.744122 2532 raft_consensus.cc:348] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:19.744233 2532 raft_consensus.cc:374] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:19.744257 2532 raft_consensus.cc:729] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.744376 2532 consensus_queue.cc:260] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:19.744649 2532 ts_tablet_manager.cc:1398] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:19.744812 2532 tablet_bootstrap.cc:492] T 1b2fc2ad5cd24a128562714712e89d89 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:19.745716 2532 tablet_bootstrap.cc:654] T 1b2fc2ad5cd24a128562714712e89d89 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:19.745755 2647 raft_consensus.cc:348] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:19.745877 2647 raft_consensus.cc:374] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:19.745910 2647 raft_consensus.cc:729] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.746048 2647 consensus_queue.cc:260] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:19.747069 2647 ts_tablet_manager.cc:1398] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20240502 20:46:19.748340 2532 tablet_bootstrap.cc:492] T 1b2fc2ad5cd24a128562714712e89d89 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:19.748422 2532 ts_tablet_manager.cc:1367] T 1b2fc2ad5cd24a128562714712e89d89 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:19.748456 2647 tablet_bootstrap.cc:492] T 1b2fc2ad5cd24a128562714712e89d89 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:19.748848 2532 raft_consensus.cc:348] T 1b2fc2ad5cd24a128562714712e89d89 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:19.748944 2532 raft_consensus.cc:374] T 1b2fc2ad5cd24a128562714712e89d89 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:19.748972 2532 raft_consensus.cc:729] T 1b2fc2ad5cd24a128562714712e89d89 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.749086 2532 consensus_queue.cc:260] T 1b2fc2ad5cd24a128562714712e89d89 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:19.749318 2532 ts_tablet_manager.cc:1398] T 1b2fc2ad5cd24a128562714712e89d89 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:19.749334 2647 tablet_bootstrap.cc:654] T 1b2fc2ad5cd24a128562714712e89d89 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:19.749468 2532 tablet_bootstrap.cc:492] T 8d5cca35ba4b47748235d1463ec161d6 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:19.750245 2532 tablet_bootstrap.cc:654] T 8d5cca35ba4b47748235d1463ec161d6 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:19.750376 2374 tablet_bootstrap.cc:492] T 1b2fc2ad5cd24a128562714712e89d89 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:19.750506 2374 ts_tablet_manager.cc:1367] T 1b2fc2ad5cd24a128562714712e89d89 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 20:46:19.750989 2374 raft_consensus.cc:348] T 1b2fc2ad5cd24a128562714712e89d89 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:19.751101 2374 raft_consensus.cc:374] T 1b2fc2ad5cd24a128562714712e89d89 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:19.751132 2374 raft_consensus.cc:729] T 1b2fc2ad5cd24a128562714712e89d89 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.751250 2374 consensus_queue.cc:260] T 1b2fc2ad5cd24a128562714712e89d89 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:19.751569 2374 ts_tablet_manager.cc:1398] T 1b2fc2ad5cd24a128562714712e89d89 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:19.753969 2647 tablet_bootstrap.cc:492] T 1b2fc2ad5cd24a128562714712e89d89 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:19.754104 2647 ts_tablet_manager.cc:1367] T 1b2fc2ad5cd24a128562714712e89d89 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 20:46:19.754591 2647 raft_consensus.cc:348] T 1b2fc2ad5cd24a128562714712e89d89 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:19.754729 2647 raft_consensus.cc:374] T 1b2fc2ad5cd24a128562714712e89d89 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:19.754772 2647 raft_consensus.cc:729] T 1b2fc2ad5cd24a128562714712e89d89 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.754925 2647 consensus_queue.cc:260] T 1b2fc2ad5cd24a128562714712e89d89 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:19.755208 2647 ts_tablet_manager.cc:1398] T 1b2fc2ad5cd24a128562714712e89d89 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:19.756635 2532 tablet_bootstrap.cc:492] T 8d5cca35ba4b47748235d1463ec161d6 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:19.756727 2532 ts_tablet_manager.cc:1367] T 8d5cca35ba4b47748235d1463ec161d6 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 20:46:19.757184 2532 raft_consensus.cc:348] T 8d5cca35ba4b47748235d1463ec161d6 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:19.757297 2532 raft_consensus.cc:374] T 8d5cca35ba4b47748235d1463ec161d6 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:19.757324 2532 raft_consensus.cc:729] T 8d5cca35ba4b47748235d1463ec161d6 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:19.757429 2532 consensus_queue.cc:260] T 8d5cca35ba4b47748235d1463ec161d6 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:19.757674 2532 ts_tablet_manager.cc:1398] T 8d5cca35ba4b47748235d1463ec161d6 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:19.765942 29035 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:19.771310 2583 tablet.cc:1722] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:19.772099 2590 tablet.cc:1722] T ca4d881c6ab943b99d43dd775b9d7281 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:19.772461 2584 tablet.cc:1722] T ca4d881c6ab943b99d43dd775b9d7281 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:19.777683 29039 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=1422daa42e834e10bc5866663d406d10] alter complete (version 3)
I20240502 20:46:19.796085 2578 raft_consensus.cc:482] T 8d5cca35ba4b47748235d1463ec161d6 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:19.796236 2578 raft_consensus.cc:504] T 8d5cca35ba4b47748235d1463ec161d6 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:19.796687 2578 leader_election.cc:288] T 8d5cca35ba4b47748235d1463ec161d6 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:19.797158 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d5cca35ba4b47748235d1463ec161d6" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:19.797274 31757 raft_consensus.cc:2454] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:19.797453 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d5cca35ba4b47748235d1463ec161d6" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:19.797578 31559 leader_election.cc:302] T 8d5cca35ba4b47748235d1463ec161d6 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:19.797596 31495 raft_consensus.cc:2454] T 8d5cca35ba4b47748235d1463ec161d6 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:19.797729 2578 raft_consensus.cc:2781] T 8d5cca35ba4b47748235d1463ec161d6 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:19.797780 2578 raft_consensus.cc:482] T 8d5cca35ba4b47748235d1463ec161d6 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:19.797813 2578 raft_consensus.cc:3037] T 8d5cca35ba4b47748235d1463ec161d6 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:19.798919 2578 raft_consensus.cc:504] T 8d5cca35ba4b47748235d1463ec161d6 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:19.799743 2578 leader_election.cc:288] T 8d5cca35ba4b47748235d1463ec161d6 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Requested vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:19.800035 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d5cca35ba4b47748235d1463ec161d6" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:19.800159 31495 raft_consensus.cc:3037] T 8d5cca35ba4b47748235d1463ec161d6 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:19.800345 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d5cca35ba4b47748235d1463ec161d6" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:19.800457 31757 raft_consensus.cc:3037] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:19.801177 31495 raft_consensus.cc:2454] T 8d5cca35ba4b47748235d1463ec161d6 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:19.801385 31757 raft_consensus.cc:2454] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:19.801458 31561 leader_election.cc:302] T 8d5cca35ba4b47748235d1463ec161d6 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, 315915efcf2b413ca7f9d874338e1b2d; no voters:
I20240502 20:46:19.801613 2578 raft_consensus.cc:2781] T 8d5cca35ba4b47748235d1463ec161d6 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:19.801702 2578 raft_consensus.cc:686] T 8d5cca35ba4b47748235d1463ec161d6 P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Becoming Leader. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Running, Role: LEADER
I20240502 20:46:19.801851 2578 consensus_queue.cc:237] T 8d5cca35ba4b47748235d1463ec161d6 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:19.805809 29039 catalog_manager.cc:5484] T 8d5cca35ba4b47748235d1463ec161d6 P 315915efcf2b413ca7f9d874338e1b2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70). New cstate: current_term: 1 leader_uuid: "315915efcf2b413ca7f9d874338e1b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:19.825578 2578 raft_consensus.cc:482] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:19.825731 2578 raft_consensus.cc:504] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:19.826699 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5bdf4f57d2c4c828b94c7cc2e3c2847" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:19.826848 29182 raft_consensus.cc:2454] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:19.827237 31562 leader_election.cc:302] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:19.827430 2289 raft_consensus.cc:2781] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:19.827478 2289 raft_consensus.cc:482] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:19.827520 2289 raft_consensus.cc:3037] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:19.828611 2289 raft_consensus.cc:504] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:19.828964 2289 leader_election.cc:288] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Requested vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:19.829425 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5bdf4f57d2c4c828b94c7cc2e3c2847" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:19.829550 29182 raft_consensus.cc:3037] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:19.829706 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5bdf4f57d2c4c828b94c7cc2e3c2847" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:19.829847 31756 raft_consensus.cc:3037] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:19.829952 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5bdf4f57d2c4c828b94c7cc2e3c2847" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:19.830374 29182 raft_consensus.cc:2454] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:19.830742 31562 leader_election.cc:302] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:19.830889 31756 raft_consensus.cc:2454] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:19.830915 2289 raft_consensus.cc:2781] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:19.831027 2578 leader_election.cc:288] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:19.831101 2289 raft_consensus.cc:686] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Becoming Leader. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Running, Role: LEADER
I20240502 20:46:19.831259 2289 consensus_queue.cc:237] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:19.833595 29033 catalog_manager.cc:5484] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 315915efcf2b413ca7f9d874338e1b2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70). New cstate: current_term: 1 leader_uuid: "315915efcf2b413ca7f9d874338e1b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:19.837395 29315 raft_consensus.cc:1203] T 6b675480162c4a9799db51ea4b6504a5 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 20:46:19.850603 31756 raft_consensus.cc:1203] T 58d4026c208f45d8870a2eaa1c97efab P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 20:46:19.857280 2650 raft_consensus.cc:482] T 1b2fc2ad5cd24a128562714712e89d89 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:19.857398 2650 raft_consensus.cc:504] T 1b2fc2ad5cd24a128562714712e89d89 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:19.857774 2650 leader_election.cc:288] T 1b2fc2ad5cd24a128562714712e89d89 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:19.858099 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b2fc2ad5cd24a128562714712e89d89" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:19.858126 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b2fc2ad5cd24a128562714712e89d89" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:19.858246 31625 raft_consensus.cc:2454] T 1b2fc2ad5cd24a128562714712e89d89 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:19.858253 29182 raft_consensus.cc:2454] T 1b2fc2ad5cd24a128562714712e89d89 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:19.858546 31693 leader_election.cc:302] T 1b2fc2ad5cd24a128562714712e89d89 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:19.858764 2650 raft_consensus.cc:2781] T 1b2fc2ad5cd24a128562714712e89d89 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:19.858819 2650 raft_consensus.cc:482] T 1b2fc2ad5cd24a128562714712e89d89 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:19.858851 2650 raft_consensus.cc:3037] T 1b2fc2ad5cd24a128562714712e89d89 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:19.859911 2650 raft_consensus.cc:504] T 1b2fc2ad5cd24a128562714712e89d89 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:19.860231 2650 leader_election.cc:288] T 1b2fc2ad5cd24a128562714712e89d89 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:19.860543 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b2fc2ad5cd24a128562714712e89d89" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:19.860649 29182 raft_consensus.cc:3037] T 1b2fc2ad5cd24a128562714712e89d89 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:19.861562 29182 raft_consensus.cc:2454] T 1b2fc2ad5cd24a128562714712e89d89 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:19.861809 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b2fc2ad5cd24a128562714712e89d89" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:19.861821 31693 leader_election.cc:302] T 1b2fc2ad5cd24a128562714712e89d89 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:19.861918 31625 raft_consensus.cc:3037] T 1b2fc2ad5cd24a128562714712e89d89 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:19.862609 2650 raft_consensus.cc:2781] T 1b2fc2ad5cd24a128562714712e89d89 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:19.862700 2650 raft_consensus.cc:686] T 1b2fc2ad5cd24a128562714712e89d89 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Becoming Leader. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Running, Role: LEADER
I20240502 20:46:19.862847 31625 raft_consensus.cc:2454] T 1b2fc2ad5cd24a128562714712e89d89 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:19.862834 2650 consensus_queue.cc:237] T 1b2fc2ad5cd24a128562714712e89d89 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:19.864732 29039 catalog_manager.cc:5484] T 1b2fc2ad5cd24a128562714712e89d89 P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: term changed from 0 to 1, leader changed from <none> to e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71). New cstate: current_term: 1 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:19.881330 29039 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:19.881484 29039 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:19.884714 29039 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet ca4d881c6ab943b99d43dd775b9d7281
I20240502 20:46:19.885484 31602 tablet_service.cc:1514] Processing DeleteTablet for tablet ca4d881c6ab943b99d43dd775b9d7281 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:19 UTC) from {username='slave'} at 127.0.0.1:58346
I20240502 20:46:19.885581 31737 tablet_service.cc:1514] Processing DeleteTablet for tablet ca4d881c6ab943b99d43dd775b9d7281 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:19 UTC) from {username='slave'} at 127.0.0.1:33864
I20240502 20:46:19.885944 31868 tablet_service.cc:1514] Processing DeleteTablet for tablet ca4d881c6ab943b99d43dd775b9d7281 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:19 UTC) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:19.887097 2716 tablet_replica.cc:330] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:19.887368 29163 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:19.890889 29293 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:19.892913 31626 raft_consensus.cc:1203] T d4a2c7b197f948a0b236d5e05e2749b9 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:19.899245 29557 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:19.900130 31626 raft_consensus.cc:1203] T 4a91eafc5ecf45b09f60d712884b7c79 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:19.901685 31868 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:19.905421 2714 tablet_replica.cc:330] T ca4d881c6ab943b99d43dd775b9d7281 P 315915efcf2b413ca7f9d874338e1b2d: stopping tablet replica
I20240502 20:46:19.905571 2714 raft_consensus.cc:2229] T ca4d881c6ab943b99d43dd775b9d7281 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:19.905834 31737 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:19.906525 2714 raft_consensus.cc:2258] T ca4d881c6ab943b99d43dd775b9d7281 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:19.906778 2715 tablet_replica.cc:330] T ca4d881c6ab943b99d43dd775b9d7281 P e7ab6b9053664ee5aa71c7978e26fe6a: stopping tablet replica
I20240502 20:46:19.907024 2715 raft_consensus.cc:2229] T ca4d881c6ab943b99d43dd775b9d7281 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:19.907203 2715 raft_consensus.cc:2258] T ca4d881c6ab943b99d43dd775b9d7281 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:19.907917 2715 ts_tablet_manager.cc:1875] T ca4d881c6ab943b99d43dd775b9d7281 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:19.907915 2714 ts_tablet_manager.cc:1875] T ca4d881c6ab943b99d43dd775b9d7281 P 315915efcf2b413ca7f9d874338e1b2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:19.910053 2715 ts_tablet_manager.cc:1888] T ca4d881c6ab943b99d43dd775b9d7281 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:19.910053 2714 ts_tablet_manager.cc:1888] T ca4d881c6ab943b99d43dd775b9d7281 P 315915efcf2b413ca7f9d874338e1b2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:19.910122 2714 log.cc:1196] T ca4d881c6ab943b99d43dd775b9d7281 P 315915efcf2b413ca7f9d874338e1b2d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal/wals/ca4d881c6ab943b99d43dd775b9d7281
I20240502 20:46:19.910238 2715 log.cc:1196] T ca4d881c6ab943b99d43dd775b9d7281 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/wals/ca4d881c6ab943b99d43dd775b9d7281
I20240502 20:46:19.910567 2715 ts_tablet_manager.cc:1909] T ca4d881c6ab943b99d43dd775b9d7281 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting consensus metadata
I20240502 20:46:19.912065 29026 catalog_manager.cc:4830] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): tablet ca4d881c6ab943b99d43dd775b9d7281 (table rebalancer_dynamic_table_new_name [id=1422daa42e834e10bc5866663d406d10]) successfully deleted
I20240502 20:46:19.913046 31475 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:19.913898 2714 ts_tablet_manager.cc:1909] T ca4d881c6ab943b99d43dd775b9d7281 P 315915efcf2b413ca7f9d874338e1b2d: Deleting consensus metadata
I20240502 20:46:19.914620 29027 catalog_manager.cc:4830] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): tablet ca4d881c6ab943b99d43dd775b9d7281 (table rebalancer_dynamic_table_new_name [id=1422daa42e834e10bc5866663d406d10]) successfully deleted
I20240502 20:46:19.915591 29424 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:19.930701 31602 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:19.931646 31887 raft_consensus.cc:1203] T a2bf196bcab440c48aa4ea893997573b P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:19.945228 2716 raft_consensus.cc:2229] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:19.947561 2716 raft_consensus.cc:2258] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:19.948195 2716 ts_tablet_manager.cc:1875] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:19.950043 2716 ts_tablet_manager.cc:1888] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:19.950117 2716 log.cc:1196] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/ca4d881c6ab943b99d43dd775b9d7281
I20240502 20:46:19.957965 2716 ts_tablet_manager.cc:1909] T ca4d881c6ab943b99d43dd775b9d7281 P e6b60803d0764d228c863e2da65e3691: Deleting consensus metadata
I20240502 20:46:19.958774 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet ca4d881c6ab943b99d43dd775b9d7281 (table rebalancer_dynamic_table_new_name [id=1422daa42e834e10bc5866663d406d10]) successfully deleted
I20240502 20:46:19.986263 31626 consensus_queue.cc:237] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:19.986701 29039 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:19.987737 31756 raft_consensus.cc:1261] T 58d4026c208f45d8870a2eaa1c97efab P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer 315915efcf2b413ca7f9d874338e1b2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:19.988263 2289 consensus_queue.cc:1035] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:19.988746 29577 raft_consensus.cc:1261] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 315915efcf2b413ca7f9d874338e1b2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:19.989948 2289 consensus_queue.cc:1035] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:19.992069 2700 raft_consensus.cc:2932] T 58d4026c208f45d8870a2eaa1c97efab P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:19.992168 29577 raft_consensus.cc:2932] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:19.994678 29033 catalog_manager.cc:5484] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: config changed from index 2 to 3, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) evicted. New cstate: current_term: 1 leader_uuid: "315915efcf2b413ca7f9d874338e1b2d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:19.994930 31756 raft_consensus.cc:2932] T 58d4026c208f45d8870a2eaa1c97efab P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:19.995867 31868 tablet_service.cc:1467] Processing CreateTablet for tablet da129a8246bc4f20843bc1c15f18a36f (DEFAULT_TABLE table=rebalancer_dynamic_table [id=479e190c46ae4ee2a1bffd01de0533cd]), partition=
I20240502 20:46:19.995865 31602 tablet_service.cc:1467] Processing CreateTablet for tablet da129a8246bc4f20843bc1c15f18a36f (DEFAULT_TABLE table=rebalancer_dynamic_table [id=479e190c46ae4ee2a1bffd01de0533cd]), partition=
I20240502 20:46:19.996114 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da129a8246bc4f20843bc1c15f18a36f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:19.996114 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da129a8246bc4f20843bc1c15f18a36f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:19.996526 31475 tablet_service.cc:1467] Processing CreateTablet for tablet da129a8246bc4f20843bc1c15f18a36f (DEFAULT_TABLE table=rebalancer_dynamic_table [id=479e190c46ae4ee2a1bffd01de0533cd]), partition=
I20240502 20:46:19.996752 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da129a8246bc4f20843bc1c15f18a36f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:19.998373 2532 tablet_bootstrap.cc:492] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:19.998796 2645 tablet_bootstrap.cc:492] T da129a8246bc4f20843bc1c15f18a36f P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:19.999362 2532 tablet_bootstrap.cc:654] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:19.999650 2645 tablet_bootstrap.cc:654] T da129a8246bc4f20843bc1c15f18a36f P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:20.000993 2736 tablet_bootstrap.cc:492] T da129a8246bc4f20843bc1c15f18a36f P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:20.002051 2736 tablet_bootstrap.cc:654] T da129a8246bc4f20843bc1c15f18a36f P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:20.002604 2645 tablet_bootstrap.cc:492] T da129a8246bc4f20843bc1c15f18a36f P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:20.002718 2645 ts_tablet_manager.cc:1367] T da129a8246bc4f20843bc1c15f18a36f P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:20.003201 2645 raft_consensus.cc:348] T da129a8246bc4f20843bc1c15f18a36f P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:20.003310 2645 raft_consensus.cc:374] T da129a8246bc4f20843bc1c15f18a36f P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:20.003340 2645 raft_consensus.cc:729] T da129a8246bc4f20843bc1c15f18a36f P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:20.003471 2645 consensus_queue.cc:260] T da129a8246bc4f20843bc1c15f18a36f P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:20.003634 29163 tablet_service.cc:1514] Processing DeleteTablet for tablet 58d4026c208f45d8870a2eaa1c97efab with delete_type TABLET_DATA_TOMBSTONED (TS 7796ea3fba294d17a34386873969431d not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:20.003777 2645 ts_tablet_manager.cc:1398] T da129a8246bc4f20843bc1c15f18a36f P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:20.005225 2736 tablet_bootstrap.cc:492] T da129a8246bc4f20843bc1c15f18a36f P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:20.005303 2736 ts_tablet_manager.cc:1367] T da129a8246bc4f20843bc1c15f18a36f P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:20.006001 2736 raft_consensus.cc:348] T da129a8246bc4f20843bc1c15f18a36f P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:20.006489 2736 raft_consensus.cc:374] T da129a8246bc4f20843bc1c15f18a36f P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:20.006582 2736 raft_consensus.cc:729] T da129a8246bc4f20843bc1c15f18a36f P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:20.007342 2736 consensus_queue.cc:260] T da129a8246bc4f20843bc1c15f18a36f P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:20.010857 2736 ts_tablet_manager.cc:1398] T da129a8246bc4f20843bc1c15f18a36f P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20240502 20:46:20.012892 2740 tablet_replica.cc:330] T 58d4026c208f45d8870a2eaa1c97efab P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:20.013093 2740 raft_consensus.cc:2229] T 58d4026c208f45d8870a2eaa1c97efab P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:20.013314 2740 raft_consensus.cc:2258] T 58d4026c208f45d8870a2eaa1c97efab P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:20.013912 2740 ts_tablet_manager.cc:1875] T 58d4026c208f45d8870a2eaa1c97efab P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:20.015087 2532 tablet_bootstrap.cc:492] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:20.015203 2532 ts_tablet_manager.cc:1367] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.000s
I20240502 20:46:20.015726 2532 raft_consensus.cc:348] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:20.015846 2532 raft_consensus.cc:374] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:20.015883 2532 raft_consensus.cc:729] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:20.016021 2532 consensus_queue.cc:260] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:20.016301 2532 ts_tablet_manager.cc:1398] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 20:46:20.016433 2740 ts_tablet_manager.cc:1888] T 58d4026c208f45d8870a2eaa1c97efab P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 20:46:20.016499 2740 log.cc:1196] T 58d4026c208f45d8870a2eaa1c97efab P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/58d4026c208f45d8870a2eaa1c97efab
I20240502 20:46:20.020153 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 58d4026c208f45d8870a2eaa1c97efab (table rebalancer_extra_table_0 [id=bec3bd33882044f38d797093171eeb1d]) successfully deleted
I20240502 20:46:20.181874 2578 raft_consensus.cc:482] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:20.182003 2578 raft_consensus.cc:504] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:20.182371 2578 leader_election.cc:288] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:20.182788 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da129a8246bc4f20843bc1c15f18a36f" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:20.182794 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da129a8246bc4f20843bc1c15f18a36f" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:20.182937 31888 raft_consensus.cc:2454] T da129a8246bc4f20843bc1c15f18a36f P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:20.182938 31495 raft_consensus.cc:2454] T da129a8246bc4f20843bc1c15f18a36f P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:20.183223 31561 leader_election.cc:302] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, 315915efcf2b413ca7f9d874338e1b2d; no voters:
I20240502 20:46:20.183424 2578 raft_consensus.cc:2781] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:20.183501 2578 raft_consensus.cc:482] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:20.183537 2578 raft_consensus.cc:3037] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:20.184633 2578 raft_consensus.cc:504] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:20.184995 2578 leader_election.cc:288] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Requested vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:20.185392 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da129a8246bc4f20843bc1c15f18a36f" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:20.185523 31495 raft_consensus.cc:3037] T da129a8246bc4f20843bc1c15f18a36f P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:20.185739 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da129a8246bc4f20843bc1c15f18a36f" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:20.185860 31888 raft_consensus.cc:3037] T da129a8246bc4f20843bc1c15f18a36f P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:20.186645 31495 raft_consensus.cc:2454] T da129a8246bc4f20843bc1c15f18a36f P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:20.186842 31888 raft_consensus.cc:2454] T da129a8246bc4f20843bc1c15f18a36f P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:20.187222 31560 leader_election.cc:302] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:20.187634 2578 raft_consensus.cc:2781] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:20.187719 2578 raft_consensus.cc:686] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Becoming Leader. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Running, Role: LEADER
I20240502 20:46:20.187842 2578 consensus_queue.cc:237] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:20.189769 29037 catalog_manager.cc:5484] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70). New cstate: current_term: 1 leader_uuid: "315915efcf2b413ca7f9d874338e1b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:20.217721 31475 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:20.220073 29293 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:20.222728 29163 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:20.226493 29557 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:20.240581 31737 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:20.241137 29424 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:20.243710 2578 consensus_queue.cc:1035] T 8d5cca35ba4b47748235d1463ec161d6 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:20.245708 31602 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:20.252835 31868 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:20.280575 2289 consensus_queue.cc:1035] T 8d5cca35ba4b47748235d1463ec161d6 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:20.295140 29037 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:20.296727 2578 consensus_queue.cc:1035] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:20.299543 31888 raft_consensus.cc:1261] T da129a8246bc4f20843bc1c15f18a36f P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer 315915efcf2b413ca7f9d874338e1b2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:20.301225 2289 consensus_queue.cc:1035] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:20.301978 2289 consensus_queue.cc:1035] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:20.304020 29033 catalog_manager.cc:5576] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207) does not have the latest schema for tablet da129a8246bc4f20843bc1c15f18a36f (table rebalancer_dynamic_table [id=479e190c46ae4ee2a1bffd01de0533cd]). Expected version 1 got 0
I20240502 20:46:20.306281 2650 consensus_queue.cc:1035] T 1b2fc2ad5cd24a128562714712e89d89 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:20.307080 31495 raft_consensus.cc:1261] T da129a8246bc4f20843bc1c15f18a36f P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Refusing update from remote peer 315915efcf2b413ca7f9d874338e1b2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:20.308634 2289 consensus_queue.cc:1035] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:20.309898 2819 tablet.cc:1722] T da129a8246bc4f20843bc1c15f18a36f P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:20.310319 2650 consensus_queue.cc:1035] T 1b2fc2ad5cd24a128562714712e89d89 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:20.310890 2822 tablet.cc:1722] T da129a8246bc4f20843bc1c15f18a36f P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:20.311041 2819 tablet.cc:1620] T da129a8246bc4f20843bc1c15f18a36f P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:20.312078 2818 tablet.cc:1722] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:20.312477 2818 tablet.cc:1620] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:20.312506 29037 catalog_manager.cc:5576] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887) does not have the latest schema for tablet da129a8246bc4f20843bc1c15f18a36f (table rebalancer_dynamic_table [id=479e190c46ae4ee2a1bffd01de0533cd]). Expected version 1 got 0
I20240502 20:46:20.315052 2822 tablet.cc:1620] T da129a8246bc4f20843bc1c15f18a36f P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:20.316476 29037 catalog_manager.cc:5576] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887) does not have the latest schema for tablet da129a8246bc4f20843bc1c15f18a36f (table rebalancer_dynamic_table [id=479e190c46ae4ee2a1bffd01de0533cd]). Expected version 1 got 0
I20240502 20:46:20.316903 29033 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=479e190c46ae4ee2a1bffd01de0533cd] alter complete (version 1)
I20240502 20:46:20.322371 31626 consensus_queue.cc:237] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:20.323638 31888 raft_consensus.cc:1261] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer 315915efcf2b413ca7f9d874338e1b2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:20.323987 29577 raft_consensus.cc:1261] T 03db15f1ebfb4e4ea271d6bb5c76110d P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 315915efcf2b413ca7f9d874338e1b2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:20.323990 2289 consensus_queue.cc:1035] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:20.324362 2289 consensus_queue.cc:1035] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:20.325682 2578 raft_consensus.cc:2932] T 03db15f1ebfb4e4ea271d6bb5c76110d P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:20.326246 31888 raft_consensus.cc:2932] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:20.326638 29577 raft_consensus.cc:2932] T 03db15f1ebfb4e4ea271d6bb5c76110d P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:20.330360 29033 catalog_manager.cc:5484] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 reported cstate change: config changed from index 2 to 3, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) evicted. New cstate: current_term: 1 leader_uuid: "315915efcf2b413ca7f9d874338e1b2d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:20.333115 31737 tablet_service.cc:1514] Processing DeleteTablet for tablet 03db15f1ebfb4e4ea271d6bb5c76110d with delete_type TABLET_DATA_TOMBSTONED (TS e7ab6b9053664ee5aa71c7978e26fe6a not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:33864
I20240502 20:46:20.333348 2715 tablet_replica.cc:330] T 03db15f1ebfb4e4ea271d6bb5c76110d P e7ab6b9053664ee5aa71c7978e26fe6a: stopping tablet replica
I20240502 20:46:20.333480 2715 raft_consensus.cc:2229] T 03db15f1ebfb4e4ea271d6bb5c76110d P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:20.333606 2715 raft_consensus.cc:2258] T 03db15f1ebfb4e4ea271d6bb5c76110d P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:20.333998 2715 ts_tablet_manager.cc:1875] T 03db15f1ebfb4e4ea271d6bb5c76110d P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:20.335968 2715 ts_tablet_manager.cc:1888] T 03db15f1ebfb4e4ea271d6bb5c76110d P e7ab6b9053664ee5aa71c7978e26fe6a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 20:46:20.336043 2715 log.cc:1196] T 03db15f1ebfb4e4ea271d6bb5c76110d P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/wals/03db15f1ebfb4e4ea271d6bb5c76110d
I20240502 20:46:20.337040 29026 catalog_manager.cc:4830] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): tablet 03db15f1ebfb4e4ea271d6bb5c76110d (table rebalancer_extra_table_6 [id=9659763c7b3a4e2f8ffa64e3e9c5e095]) successfully deleted
I20240502 20:46:20.390842 29042 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_22"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:20.401877 31602 tablet_service.cc:1467] Processing CreateTablet for tablet 4d11a8f31a294ffdb9a1c3c811a38e77 (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=a157eb12111441b2b4cf5d30079f7c5f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:20.402221 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d11a8f31a294ffdb9a1c3c811a38e77. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:20.402179 31606 tablet_service.cc:1467] Processing CreateTablet for tablet 2bec922ff95845d1a13c27526f689e15 (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=a157eb12111441b2b4cf5d30079f7c5f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:20.402437 31606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bec922ff95845d1a13c27526f689e15. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:20.402480 31868 tablet_service.cc:1467] Processing CreateTablet for tablet 4d11a8f31a294ffdb9a1c3c811a38e77 (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=a157eb12111441b2b4cf5d30079f7c5f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:20.402767 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d11a8f31a294ffdb9a1c3c811a38e77. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:20.404804 31605 tablet_service.cc:1467] Processing CreateTablet for tablet 63487637b76a4de89ac3f53c707f074b (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=a157eb12111441b2b4cf5d30079f7c5f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:20.404839 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 63487637b76a4de89ac3f53c707f074b (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=a157eb12111441b2b4cf5d30079f7c5f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:20.404953 29557 tablet_service.cc:1467] Processing CreateTablet for tablet 2bec922ff95845d1a13c27526f689e15 (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=a157eb12111441b2b4cf5d30079f7c5f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:20.405043 31605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63487637b76a4de89ac3f53c707f074b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:20.405057 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63487637b76a4de89ac3f53c707f074b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:20.405198 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bec922ff95845d1a13c27526f689e15. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:20.407007 31475 tablet_service.cc:1467] Processing CreateTablet for tablet 4d11a8f31a294ffdb9a1c3c811a38e77 (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=a157eb12111441b2b4cf5d30079f7c5f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:20.407287 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d11a8f31a294ffdb9a1c3c811a38e77. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:20.407481 2532 tablet_bootstrap.cc:492] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:20.407820 29426 tablet_service.cc:1467] Processing CreateTablet for tablet 63487637b76a4de89ac3f53c707f074b (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=a157eb12111441b2b4cf5d30079f7c5f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:20.408082 29426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63487637b76a4de89ac3f53c707f074b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:20.408413 2532 tablet_bootstrap.cc:654] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:20.409219 29424 tablet_service.cc:1467] Processing CreateTablet for tablet 2bec922ff95845d1a13c27526f689e15 (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=a157eb12111441b2b4cf5d30079f7c5f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:20.409374 2645 tablet_bootstrap.cc:492] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:20.409449 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bec922ff95845d1a13c27526f689e15. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:20.410342 2645 tablet_bootstrap.cc:654] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:20.410835 2873 tablet_bootstrap.cc:492] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:20.411300 2532 tablet_bootstrap.cc:492] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:20.411396 2532 ts_tablet_manager.cc:1367] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:20.411410 2736 tablet_bootstrap.cc:492] T 4d11a8f31a294ffdb9a1c3c811a38e77 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:20.412045 2532 raft_consensus.cc:348] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:20.412161 2532 raft_consensus.cc:374] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:20.412192 2532 raft_consensus.cc:729] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:20.412369 2736 tablet_bootstrap.cc:654] T 4d11a8f31a294ffdb9a1c3c811a38e77 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:20.412482 2532 consensus_queue.cc:260] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:20.412463 2875 tablet_bootstrap.cc:492] T 63487637b76a4de89ac3f53c707f074b P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:20.413007 2532 ts_tablet_manager.cc:1398] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:20.413267 2532 tablet_bootstrap.cc:492] T 63487637b76a4de89ac3f53c707f074b P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:20.413552 2645 tablet_bootstrap.cc:492] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:20.413663 2645 ts_tablet_manager.cc:1367] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 20:46:20.414096 2645 raft_consensus.cc:348] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:20.414201 2645 raft_consensus.cc:374] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:20.414232 2645 raft_consensus.cc:729] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:20.414342 2645 consensus_queue.cc:260] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:20.414634 2645 ts_tablet_manager.cc:1398] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:20.419188 29035 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:20.421494 2532 tablet_bootstrap.cc:654] T 63487637b76a4de89ac3f53c707f074b P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:20.422214 2736 tablet_bootstrap.cc:492] T 4d11a8f31a294ffdb9a1c3c811a38e77 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:20.422307 2736 ts_tablet_manager.cc:1367] T 4d11a8f31a294ffdb9a1c3c811a38e77 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 20:46:20.422827 2736 raft_consensus.cc:348] T 4d11a8f31a294ffdb9a1c3c811a38e77 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:20.422955 2736 raft_consensus.cc:374] T 4d11a8f31a294ffdb9a1c3c811a38e77 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:20.422994 2736 raft_consensus.cc:729] T 4d11a8f31a294ffdb9a1c3c811a38e77 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:20.423115 2736 consensus_queue.cc:260] T 4d11a8f31a294ffdb9a1c3c811a38e77 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:20.423401 2736 ts_tablet_manager.cc:1398] T 4d11a8f31a294ffdb9a1c3c811a38e77 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:20.423521 2875 tablet_bootstrap.cc:654] T 63487637b76a4de89ac3f53c707f074b P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:20.423597 2736 tablet_bootstrap.cc:492] T 63487637b76a4de89ac3f53c707f074b P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:20.423900 2532 tablet_bootstrap.cc:492] T 63487637b76a4de89ac3f53c707f074b P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:20.423981 2532 ts_tablet_manager.cc:1367] T 63487637b76a4de89ac3f53c707f074b P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 20:46:20.422874 2873 tablet_bootstrap.cc:654] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:20.424494 2736 tablet_bootstrap.cc:654] T 63487637b76a4de89ac3f53c707f074b P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:20.424428 2532 raft_consensus.cc:348] T 63487637b76a4de89ac3f53c707f074b P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:20.424526 2532 raft_consensus.cc:374] T 63487637b76a4de89ac3f53c707f074b P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:20.424552 2532 raft_consensus.cc:729] T 63487637b76a4de89ac3f53c707f074b P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:20.424674 2532 consensus_queue.cc:260] T 63487637b76a4de89ac3f53c707f074b P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:20.424934 2532 ts_tablet_manager.cc:1398] T 63487637b76a4de89ac3f53c707f074b P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:20.425117 2532 tablet_bootstrap.cc:492] T 2bec922ff95845d1a13c27526f689e15 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:20.426026 2532 tablet_bootstrap.cc:654] T 2bec922ff95845d1a13c27526f689e15 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:20.426369 2873 tablet_bootstrap.cc:492] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:20.426484 2873 ts_tablet_manager.cc:1367] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.000s
I20240502 20:46:20.427153 2875 tablet_bootstrap.cc:492] T 63487637b76a4de89ac3f53c707f074b P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:20.427313 2875 ts_tablet_manager.cc:1367] T 63487637b76a4de89ac3f53c707f074b P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.002s
I20240502 20:46:20.427827 2875 raft_consensus.cc:348] T 63487637b76a4de89ac3f53c707f074b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:20.428011 2875 raft_consensus.cc:374] T 63487637b76a4de89ac3f53c707f074b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:20.428092 2875 raft_consensus.cc:729] T 63487637b76a4de89ac3f53c707f074b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:20.428452 2875 consensus_queue.cc:260] T 63487637b76a4de89ac3f53c707f074b P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:20.428818 2875 ts_tablet_manager.cc:1398] T 63487637b76a4de89ac3f53c707f074b P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:20.429050 2875 tablet_bootstrap.cc:492] T 2bec922ff95845d1a13c27526f689e15 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:20.430542 2875 tablet_bootstrap.cc:654] T 2bec922ff95845d1a13c27526f689e15 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:20.433670 2818 tablet.cc:1722] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:20.433984 2818 tablet.cc:1620] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:20.436560 2736 tablet_bootstrap.cc:492] T 63487637b76a4de89ac3f53c707f074b P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:20.436661 2736 ts_tablet_manager.cc:1367] T 63487637b76a4de89ac3f53c707f074b P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 20:46:20.437129 2736 raft_consensus.cc:348] T 63487637b76a4de89ac3f53c707f074b P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:20.437244 2736 raft_consensus.cc:374] T 63487637b76a4de89ac3f53c707f074b P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:20.437278 2736 raft_consensus.cc:729] T 63487637b76a4de89ac3f53c707f074b P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:20.437386 2736 consensus_queue.cc:260] T 63487637b76a4de89ac3f53c707f074b P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:20.437676 2736 ts_tablet_manager.cc:1398] T 63487637b76a4de89ac3f53c707f074b P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:20.438606 2819 tablet.cc:1722] T da129a8246bc4f20843bc1c15f18a36f P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:20.438892 2819 tablet.cc:1620] T da129a8246bc4f20843bc1c15f18a36f P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:20.440178 2822 tablet.cc:1722] T da129a8246bc4f20843bc1c15f18a36f P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:20.440477 2822 tablet.cc:1620] T da129a8246bc4f20843bc1c15f18a36f P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:20.441156 29042 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=479e190c46ae4ee2a1bffd01de0533cd] alter complete (version 2)
I20240502 20:46:20.442224 2873 raft_consensus.cc:348] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:20.442360 2873 raft_consensus.cc:374] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:20.442396 2873 raft_consensus.cc:729] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:20.442595 2873 consensus_queue.cc:260] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:20.442976 2873 ts_tablet_manager.cc:1398] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.010s user 0.001s sys 0.000s
I20240502 20:46:20.445325 2875 tablet_bootstrap.cc:492] T 2bec922ff95845d1a13c27526f689e15 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:20.445362 2532 tablet_bootstrap.cc:492] T 2bec922ff95845d1a13c27526f689e15 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:20.445456 2532 ts_tablet_manager.cc:1367] T 2bec922ff95845d1a13c27526f689e15 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.001s
I20240502 20:46:20.445485 2875 ts_tablet_manager.cc:1367] T 2bec922ff95845d1a13c27526f689e15 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.002s
I20240502 20:46:20.445894 2532 raft_consensus.cc:348] T 2bec922ff95845d1a13c27526f689e15 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:20.446000 2532 raft_consensus.cc:374] T 2bec922ff95845d1a13c27526f689e15 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:20.446029 2532 raft_consensus.cc:729] T 2bec922ff95845d1a13c27526f689e15 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:20.445981 2875 raft_consensus.cc:348] T 2bec922ff95845d1a13c27526f689e15 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:20.446107 2875 raft_consensus.cc:374] T 2bec922ff95845d1a13c27526f689e15 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:20.446143 2875 raft_consensus.cc:729] T 2bec922ff95845d1a13c27526f689e15 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:20.446144 2532 consensus_queue.cc:260] T 2bec922ff95845d1a13c27526f689e15 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:20.446269 2875 consensus_queue.cc:260] T 2bec922ff95845d1a13c27526f689e15 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:20.446377 2532 ts_tablet_manager.cc:1398] T 2bec922ff95845d1a13c27526f689e15 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:20.446599 2875 ts_tablet_manager.cc:1398] T 2bec922ff95845d1a13c27526f689e15 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:20.460837 31606 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:20.462517 29293 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:20.467329 31475 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:20.468122 29557 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:20.469290 29163 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:20.469523 31867 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:20.473932 29424 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:20.493541 31737 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:20.508447 2882 raft_consensus.cc:482] T 63487637b76a4de89ac3f53c707f074b P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:20.508582 2882 raft_consensus.cc:504] T 63487637b76a4de89ac3f53c707f074b P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:20.508997 2882 leader_election.cc:288] T 63487637b76a4de89ac3f53c707f074b P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:20.509503 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63487637b76a4de89ac3f53c707f074b" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:20.509647 29446 raft_consensus.cc:2454] T 63487637b76a4de89ac3f53c707f074b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:20.509932 31823 leader_election.cc:302] T 63487637b76a4de89ac3f53c707f074b P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dbcdc6ac635b44b185c3eba63f66e9d6, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:20.510327 2882 raft_consensus.cc:2781] T 63487637b76a4de89ac3f53c707f074b P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:20.510383 2882 raft_consensus.cc:482] T 63487637b76a4de89ac3f53c707f074b P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:20.510416 2882 raft_consensus.cc:3037] T 63487637b76a4de89ac3f53c707f074b P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:20.510607 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63487637b76a4de89ac3f53c707f074b" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:20.510738 31626 raft_consensus.cc:2454] T 63487637b76a4de89ac3f53c707f074b P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:20.511590 2882 raft_consensus.cc:504] T 63487637b76a4de89ac3f53c707f074b P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:20.511929 2882 leader_election.cc:288] T 63487637b76a4de89ac3f53c707f074b P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:20.512828 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63487637b76a4de89ac3f53c707f074b" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:20.512900 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63487637b76a4de89ac3f53c707f074b" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:20.512951 29446 raft_consensus.cc:3037] T 63487637b76a4de89ac3f53c707f074b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:20.513001 31626 raft_consensus.cc:3037] T 63487637b76a4de89ac3f53c707f074b P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:20.513972 29446 raft_consensus.cc:2454] T 63487637b76a4de89ac3f53c707f074b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:20.513972 31626 raft_consensus.cc:2454] T 63487637b76a4de89ac3f53c707f074b P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:20.514319 31822 leader_election.cc:302] T 63487637b76a4de89ac3f53c707f074b P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:20.514848 2882 raft_consensus.cc:2781] T 63487637b76a4de89ac3f53c707f074b P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:20.514926 2882 raft_consensus.cc:686] T 63487637b76a4de89ac3f53c707f074b P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Becoming Leader. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Running, Role: LEADER
I20240502 20:46:20.515040 2882 consensus_queue.cc:237] T 63487637b76a4de89ac3f53c707f074b P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:20.517374 29040 catalog_manager.cc:5484] T 63487637b76a4de89ac3f53c707f074b P e6b60803d0764d228c863e2da65e3691 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6b60803d0764d228c863e2da65e3691 (127.22.45.72). New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:20.553891 31757 consensus_queue.cc:237] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:20.554641 29040 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:20.555426 29182 raft_consensus.cc:1261] T f14cf6deefec46c7adb598f6d7cfd649 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer e7ab6b9053664ee5aa71c7978e26fe6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:20.555465 29577 raft_consensus.cc:1261] T f14cf6deefec46c7adb598f6d7cfd649 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer e7ab6b9053664ee5aa71c7978e26fe6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:20.555930 692 consensus_queue.cc:1035] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:20.556586 692 consensus_queue.cc:1035] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:20.559146 692 raft_consensus.cc:2932] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:20.561762 2818 tablet.cc:1722] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:20.561828 29182 raft_consensus.cc:2932] T f14cf6deefec46c7adb598f6d7cfd649 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:20.561968 29577 raft_consensus.cc:2932] T f14cf6deefec46c7adb598f6d7cfd649 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:20.564316 29040 catalog_manager.cc:5484] T f14cf6deefec46c7adb598f6d7cfd649 P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: config changed from index 2 to 3, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) evicted. New cstate: current_term: 1 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:20.565546 2822 tablet.cc:1722] T da129a8246bc4f20843bc1c15f18a36f P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:20.566411 2819 tablet.cc:1722] T da129a8246bc4f20843bc1c15f18a36f P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:20.571367 31475 tablet_service.cc:1514] Processing DeleteTablet for tablet f14cf6deefec46c7adb598f6d7cfd649 with delete_type TABLET_DATA_TOMBSTONED (TS 2a9c1af8cf814030bde56b67ca5c2fbe not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:40276
I20240502 20:46:20.571861 2917 tablet_replica.cc:330] T f14cf6deefec46c7adb598f6d7cfd649 P 2a9c1af8cf814030bde56b67ca5c2fbe: stopping tablet replica
I20240502 20:46:20.572001 2917 raft_consensus.cc:2229] T f14cf6deefec46c7adb598f6d7cfd649 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:20.572212 2917 raft_consensus.cc:2258] T f14cf6deefec46c7adb598f6d7cfd649 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:20.572698 2917 ts_tablet_manager.cc:1875] T f14cf6deefec46c7adb598f6d7cfd649 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:20.573231 2717 raft_consensus.cc:482] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:20.573340 2717 raft_consensus.cc:504] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:20.573751 2717 leader_election.cc:288] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:20.573823 29035 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=479e190c46ae4ee2a1bffd01de0533cd] alter complete (version 3)
I20240502 20:46:20.574414 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bec922ff95845d1a13c27526f689e15" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:20.574576 31626 raft_consensus.cc:2454] T 2bec922ff95845d1a13c27526f689e15 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:20.574606 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bec922ff95845d1a13c27526f689e15" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:20.574738 29446 raft_consensus.cc:2454] T 2bec922ff95845d1a13c27526f689e15 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:20.574869 29511 leader_election.cc:302] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:20.575254 2717 raft_consensus.cc:2781] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:20.575277 2917 ts_tablet_manager.cc:1888] T f14cf6deefec46c7adb598f6d7cfd649 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 20:46:20.575315 2717 raft_consensus.cc:482] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:20.575343 2917 log.cc:1196] T f14cf6deefec46c7adb598f6d7cfd649 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/wals/f14cf6deefec46c7adb598f6d7cfd649
I20240502 20:46:20.575353 2717 raft_consensus.cc:3037] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:20.575845 29028 catalog_manager.cc:4830] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): tablet f14cf6deefec46c7adb598f6d7cfd649 (table rebalancer_extra_table_10 [id=9bcf2b1be3a34dd7a9f3320c43834eda]) successfully deleted
I20240502 20:46:20.576224 2717 raft_consensus.cc:504] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:20.576519 2717 leader_election.cc:288] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:20.576824 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bec922ff95845d1a13c27526f689e15" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:20.576931 29446 raft_consensus.cc:3037] T 2bec922ff95845d1a13c27526f689e15 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:20.576956 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bec922ff95845d1a13c27526f689e15" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:20.577090 31626 raft_consensus.cc:3037] T 2bec922ff95845d1a13c27526f689e15 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:20.577813 29446 raft_consensus.cc:2454] T 2bec922ff95845d1a13c27526f689e15 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:20.577852 31626 raft_consensus.cc:2454] T 2bec922ff95845d1a13c27526f689e15 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:20.578133 29511 leader_election.cc:302] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:20.578320 2717 raft_consensus.cc:2781] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:20.578666 2717 raft_consensus.cc:686] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Becoming Leader. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Running, Role: LEADER
I20240502 20:46:20.578815 2717 consensus_queue.cc:237] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:20.580824 29040 catalog_manager.cc:5484] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: term changed from 0 to 1, leader changed from <none> to db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:20.678707 29036 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:20.678884 29036 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:20.681239 29036 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet da129a8246bc4f20843bc1c15f18a36f
I20240502 20:46:20.681773 31606 tablet_service.cc:1514] Processing DeleteTablet for tablet da129a8246bc4f20843bc1c15f18a36f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:20 UTC) from {username='slave'} at 127.0.0.1:58346
I20240502 20:46:20.681977 31475 tablet_service.cc:1514] Processing DeleteTablet for tablet da129a8246bc4f20843bc1c15f18a36f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:20 UTC) from {username='slave'} at 127.0.0.1:40276
I20240502 20:46:20.682124 2962 tablet_replica.cc:330] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d: stopping tablet replica
I20240502 20:46:20.682101 31867 tablet_service.cc:1514] Processing DeleteTablet for tablet da129a8246bc4f20843bc1c15f18a36f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:20 UTC) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:20.682166 2917 tablet_replica.cc:330] T da129a8246bc4f20843bc1c15f18a36f P 2a9c1af8cf814030bde56b67ca5c2fbe: stopping tablet replica
I20240502 20:46:20.682294 2917 raft_consensus.cc:2229] T da129a8246bc4f20843bc1c15f18a36f P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:20.682292 2962 raft_consensus.cc:2229] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:20.682435 2917 raft_consensus.cc:2258] T da129a8246bc4f20843bc1c15f18a36f P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:20.682440 2963 tablet_replica.cc:330] T da129a8246bc4f20843bc1c15f18a36f P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:20.682531 2962 raft_consensus.cc:2258] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:20.682596 2963 raft_consensus.cc:2229] T da129a8246bc4f20843bc1c15f18a36f P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:20.682713 2963 raft_consensus.cc:2258] T da129a8246bc4f20843bc1c15f18a36f P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:20.683209 2962 ts_tablet_manager.cc:1875] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:20.683302 2963 ts_tablet_manager.cc:1875] T da129a8246bc4f20843bc1c15f18a36f P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:20.683315 2917 ts_tablet_manager.cc:1875] T da129a8246bc4f20843bc1c15f18a36f P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:20.685339 2962 ts_tablet_manager.cc:1888] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:20.685415 2962 log.cc:1196] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal/wals/da129a8246bc4f20843bc1c15f18a36f
I20240502 20:46:20.685447 2917 ts_tablet_manager.cc:1888] T da129a8246bc4f20843bc1c15f18a36f P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:20.685528 2917 log.cc:1196] T da129a8246bc4f20843bc1c15f18a36f P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/wals/da129a8246bc4f20843bc1c15f18a36f
I20240502 20:46:20.685516 2963 ts_tablet_manager.cc:1888] T da129a8246bc4f20843bc1c15f18a36f P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:20.685580 2963 log.cc:1196] T da129a8246bc4f20843bc1c15f18a36f P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/da129a8246bc4f20843bc1c15f18a36f
I20240502 20:46:20.685832 2962 ts_tablet_manager.cc:1909] T da129a8246bc4f20843bc1c15f18a36f P 315915efcf2b413ca7f9d874338e1b2d: Deleting consensus metadata
I20240502 20:46:20.685856 2963 ts_tablet_manager.cc:1909] T da129a8246bc4f20843bc1c15f18a36f P e6b60803d0764d228c863e2da65e3691: Deleting consensus metadata
I20240502 20:46:20.686866 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet da129a8246bc4f20843bc1c15f18a36f (table rebalancer_dynamic_table_new_name [id=479e190c46ae4ee2a1bffd01de0533cd]) successfully deleted
I20240502 20:46:20.687040 2917 ts_tablet_manager.cc:1909] T da129a8246bc4f20843bc1c15f18a36f P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting consensus metadata
I20240502 20:46:20.687358 29027 catalog_manager.cc:4830] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): tablet da129a8246bc4f20843bc1c15f18a36f (table rebalancer_dynamic_table_new_name [id=479e190c46ae4ee2a1bffd01de0533cd]) successfully deleted
I20240502 20:46:20.687634 29028 catalog_manager.cc:4830] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): tablet da129a8246bc4f20843bc1c15f18a36f (table rebalancer_dynamic_table_new_name [id=479e190c46ae4ee2a1bffd01de0533cd]) successfully deleted
I20240502 20:46:20.695078 2821 raft_consensus.cc:482] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:20.695191 2821 raft_consensus.cc:504] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:20.695616 2821 leader_election.cc:288] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:20.695940 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d11a8f31a294ffdb9a1c3c811a38e77" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:20.696060 31626 raft_consensus.cc:2454] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:20.696010 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d11a8f31a294ffdb9a1c3c811a38e77" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:20.696161 31888 raft_consensus.cc:2454] T 4d11a8f31a294ffdb9a1c3c811a38e77 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:20.696328 31429 leader_election.cc:302] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, 315915efcf2b413ca7f9d874338e1b2d; no voters:
I20240502 20:46:20.696480 2821 raft_consensus.cc:2781] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:20.696537 2821 raft_consensus.cc:482] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:20.696573 2821 raft_consensus.cc:3037] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:20.697365 2821 raft_consensus.cc:504] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:20.697649 2821 leader_election.cc:288] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:20.697957 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d11a8f31a294ffdb9a1c3c811a38e77" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:20.697983 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d11a8f31a294ffdb9a1c3c811a38e77" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:20.698065 31888 raft_consensus.cc:3037] T 4d11a8f31a294ffdb9a1c3c811a38e77 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:20.698107 31626 raft_consensus.cc:3037] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:20.698851 31888 raft_consensus.cc:2454] T 4d11a8f31a294ffdb9a1c3c811a38e77 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:20.699088 31429 leader_election.cc:302] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:20.699081 31626 raft_consensus.cc:2454] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:20.699223 2821 raft_consensus.cc:2781] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:20.699294 2821 raft_consensus.cc:686] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Becoming Leader. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Running, Role: LEADER
I20240502 20:46:20.699424 2821 consensus_queue.cc:237] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:20.701745 29036 catalog_manager.cc:5484] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69). New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:20.717342 29163 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:20.722594 31606 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:20.738678 31867 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:20.738962 29293 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:20.742494 31737 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:20.745669 31475 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:20.751287 29424 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:20.755728 29557 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:20.783006 29039 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:20.789734 31867 tablet_service.cc:1467] Processing CreateTablet for tablet e6c4249d94864dd2bdf4df4947d07eb4 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=e73e91b41c7348b5bf1cdb369d78588c]), partition=
I20240502 20:46:20.789752 31737 tablet_service.cc:1467] Processing CreateTablet for tablet e6c4249d94864dd2bdf4df4947d07eb4 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=e73e91b41c7348b5bf1cdb369d78588c]), partition=
I20240502 20:46:20.789831 31475 tablet_service.cc:1467] Processing CreateTablet for tablet e6c4249d94864dd2bdf4df4947d07eb4 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=e73e91b41c7348b5bf1cdb369d78588c]), partition=
I20240502 20:46:20.790033 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6c4249d94864dd2bdf4df4947d07eb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:20.790037 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6c4249d94864dd2bdf4df4947d07eb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:20.790038 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6c4249d94864dd2bdf4df4947d07eb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:20.792011 2736 tablet_bootstrap.cc:492] T e6c4249d94864dd2bdf4df4947d07eb4 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:20.792352 2645 tablet_bootstrap.cc:492] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:20.792994 2736 tablet_bootstrap.cc:654] T e6c4249d94864dd2bdf4df4947d07eb4 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:20.793274 2645 tablet_bootstrap.cc:654] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:20.794162 2976 tablet_bootstrap.cc:492] T e6c4249d94864dd2bdf4df4947d07eb4 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:20.794710 2645 tablet_bootstrap.cc:492] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:20.794775 2645 ts_tablet_manager.cc:1367] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:20.795212 2645 raft_consensus.cc:348] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:20.795317 2645 raft_consensus.cc:374] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:20.795348 2645 raft_consensus.cc:729] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:20.795451 2645 consensus_queue.cc:260] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:20.795728 2645 ts_tablet_manager.cc:1398] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:20.796754 2736 tablet_bootstrap.cc:492] T e6c4249d94864dd2bdf4df4947d07eb4 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:20.796850 2736 ts_tablet_manager.cc:1367] T e6c4249d94864dd2bdf4df4947d07eb4 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 20:46:20.797297 2976 tablet_bootstrap.cc:654] T e6c4249d94864dd2bdf4df4947d07eb4 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:20.797317 2736 raft_consensus.cc:348] T e6c4249d94864dd2bdf4df4947d07eb4 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:20.797420 2736 raft_consensus.cc:374] T e6c4249d94864dd2bdf4df4947d07eb4 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:20.797448 2736 raft_consensus.cc:729] T e6c4249d94864dd2bdf4df4947d07eb4 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:20.797556 2736 consensus_queue.cc:260] T e6c4249d94864dd2bdf4df4947d07eb4 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:20.797811 2736 ts_tablet_manager.cc:1398] T e6c4249d94864dd2bdf4df4947d07eb4 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:20.801429 2976 tablet_bootstrap.cc:492] T e6c4249d94864dd2bdf4df4947d07eb4 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:20.801529 2976 ts_tablet_manager.cc:1367] T e6c4249d94864dd2bdf4df4947d07eb4 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 20:46:20.801981 2976 raft_consensus.cc:348] T e6c4249d94864dd2bdf4df4947d07eb4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:20.802096 2976 raft_consensus.cc:374] T e6c4249d94864dd2bdf4df4947d07eb4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:20.802139 2976 raft_consensus.cc:729] T e6c4249d94864dd2bdf4df4947d07eb4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:20.802280 2976 consensus_queue.cc:260] T e6c4249d94864dd2bdf4df4947d07eb4 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:20.802661 2976 ts_tablet_manager.cc:1398] T e6c4249d94864dd2bdf4df4947d07eb4 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:20.818986 29577 consensus_queue.cc:237] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:20.820300 31626 raft_consensus.cc:1261] T fbd2bd8e142741ba864e2f99610b39d6 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:20.820704 29182 raft_consensus.cc:1261] T fbd2bd8e142741ba864e2f99610b39d6 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:20.820758 2918 consensus_queue.cc:1035] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:20.821115 2717 consensus_queue.cc:1035] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:20.822921 2717 raft_consensus.cc:2932] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:20.823256 31626 raft_consensus.cc:2932] T fbd2bd8e142741ba864e2f99610b39d6 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:20.825513 29036 catalog_manager.cc:5484] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:20.827423 29182 raft_consensus.cc:2932] T fbd2bd8e142741ba864e2f99610b39d6 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:20.830060 29424 tablet_service.cc:1514] Processing DeleteTablet for tablet fbd2bd8e142741ba864e2f99610b39d6 with delete_type TABLET_DATA_TOMBSTONED (TS dbcdc6ac635b44b185c3eba63f66e9d6 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:20.830433 2999 tablet_replica.cc:330] T fbd2bd8e142741ba864e2f99610b39d6 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:20.830633 2999 raft_consensus.cc:2229] T fbd2bd8e142741ba864e2f99610b39d6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:20.831606 2999 raft_consensus.cc:2258] T fbd2bd8e142741ba864e2f99610b39d6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:20.832294 2999 ts_tablet_manager.cc:1875] T fbd2bd8e142741ba864e2f99610b39d6 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:20.834043 2999 ts_tablet_manager.cc:1888] T fbd2bd8e142741ba864e2f99610b39d6 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 20:46:20.834113 2999 log.cc:1196] T fbd2bd8e142741ba864e2f99610b39d6 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/fbd2bd8e142741ba864e2f99610b39d6
I20240502 20:46:20.834615 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet fbd2bd8e142741ba864e2f99610b39d6 (table rebalance_test_table_15 [id=02149b4452244d3fbab1649adf739573]) successfully deleted
I20240502 20:46:20.979163 2082 consensus_queue.cc:1035] T 63487637b76a4de89ac3f53c707f074b P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:20.982785 2882 consensus_queue.cc:1035] T 63487637b76a4de89ac3f53c707f074b P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:21.017796 2821 raft_consensus.cc:482] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:21.017926 2821 raft_consensus.cc:504] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:21.018344 2821 leader_election.cc:288] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:21.018736 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6c4249d94864dd2bdf4df4947d07eb4" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:21.018752 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6c4249d94864dd2bdf4df4947d07eb4" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:21.018872 31888 raft_consensus.cc:2454] T e6c4249d94864dd2bdf4df4947d07eb4 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:21.018884 31757 raft_consensus.cc:2454] T e6c4249d94864dd2bdf4df4947d07eb4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:21.019152 31429 leader_election.cc:302] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:21.019320 2821 raft_consensus.cc:2781] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:21.019383 2821 raft_consensus.cc:482] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:21.019429 2821 raft_consensus.cc:3037] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:21.020287 2821 raft_consensus.cc:504] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:21.020597 2821 leader_election.cc:288] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Requested vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:21.020893 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6c4249d94864dd2bdf4df4947d07eb4" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:21.020936 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6c4249d94864dd2bdf4df4947d07eb4" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:21.021005 31888 raft_consensus.cc:3037] T e6c4249d94864dd2bdf4df4947d07eb4 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:21.021023 31757 raft_consensus.cc:3037] T e6c4249d94864dd2bdf4df4947d07eb4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:21.021823 31757 raft_consensus.cc:2454] T e6c4249d94864dd2bdf4df4947d07eb4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:21.021894 31888 raft_consensus.cc:2454] T e6c4249d94864dd2bdf4df4947d07eb4 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:21.022099 31428 leader_election.cc:302] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:21.022289 2821 raft_consensus.cc:2781] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:21.022384 2821 raft_consensus.cc:686] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Becoming Leader. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Running, Role: LEADER
I20240502 20:46:21.022542 2821 consensus_queue.cc:237] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:21.025604 29039 catalog_manager.cc:5484] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69). New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:21.059540 29557 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:21.059911 2998 consensus_queue.cc:1035] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:21.060451 31606 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:21.066670 2998 consensus_queue.cc:1035] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:21.067004 29293 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:21.068688 29163 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:21.070434 31867 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:21.070824 29424 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:21.071076 31475 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:21.073295 31737 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:21.120765 2800 consensus_queue.cc:1035] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:21.127149 2800 consensus_queue.cc:1035] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:21.135154 29036 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:21.139158 31757 raft_consensus.cc:1261] T e6c4249d94864dd2bdf4df4947d07eb4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:21.139513 31888 raft_consensus.cc:1261] T e6c4249d94864dd2bdf4df4947d07eb4 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:21.139586 2800 consensus_queue.cc:1035] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:21.139952 2800 consensus_queue.cc:1035] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:21.142030 29036 catalog_manager.cc:5576] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787) does not have the latest schema for tablet e6c4249d94864dd2bdf4df4947d07eb4 (table rebalancer_dynamic_table [id=e73e91b41c7348b5bf1cdb369d78588c]). Expected version 1 got 0
I20240502 20:46:21.142668 29039 catalog_manager.cc:5576] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207) does not have the latest schema for tablet e6c4249d94864dd2bdf4df4947d07eb4 (table rebalancer_dynamic_table [id=e73e91b41c7348b5bf1cdb369d78588c]). Expected version 1 got 0
I20240502 20:46:21.143123 3083 tablet.cc:1722] T e6c4249d94864dd2bdf4df4947d07eb4 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:21.143571 3082 tablet.cc:1722] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:21.144181 3082 tablet.cc:1620] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:21.144232 3083 tablet.cc:1620] T e6c4249d94864dd2bdf4df4947d07eb4 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:21.146778 29038 catalog_manager.cc:5576] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787) does not have the latest schema for tablet e6c4249d94864dd2bdf4df4947d07eb4 (table rebalancer_dynamic_table [id=e73e91b41c7348b5bf1cdb369d78588c]). Expected version 1 got 0
I20240502 20:46:21.147614 29039 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=e73e91b41c7348b5bf1cdb369d78588c] alter complete (version 1)
I20240502 20:46:21.149515 3084 tablet.cc:1722] T e6c4249d94864dd2bdf4df4947d07eb4 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:21.149871 3084 tablet.cc:1620] T e6c4249d94864dd2bdf4df4947d07eb4 P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:21.152971 29315 consensus_queue.cc:237] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:21.154119 31495 raft_consensus.cc:1261] T 689b7368353a430dbffa1cf44f9b5949 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:21.154284 29577 raft_consensus.cc:1261] T 689b7368353a430dbffa1cf44f9b5949 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:21.154575 3086 consensus_queue.cc:1035] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:21.154860 2104 consensus_queue.cc:1035] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:21.156469 2104 raft_consensus.cc:2932] T 689b7368353a430dbffa1cf44f9b5949 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:21.156776 31495 raft_consensus.cc:2932] T 689b7368353a430dbffa1cf44f9b5949 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:21.156957 29577 raft_consensus.cc:2932] T 689b7368353a430dbffa1cf44f9b5949 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:21.159247 29036 catalog_manager.cc:5484] T 689b7368353a430dbffa1cf44f9b5949 P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:21.162359 29424 tablet_service.cc:1514] Processing DeleteTablet for tablet 689b7368353a430dbffa1cf44f9b5949 with delete_type TABLET_DATA_TOMBSTONED (TS dbcdc6ac635b44b185c3eba63f66e9d6 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:21.162644 2999 tablet_replica.cc:330] T 689b7368353a430dbffa1cf44f9b5949 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:21.162819 2999 raft_consensus.cc:2229] T 689b7368353a430dbffa1cf44f9b5949 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:21.162967 2999 raft_consensus.cc:2258] T 689b7368353a430dbffa1cf44f9b5949 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:21.163534 2999 ts_tablet_manager.cc:1875] T 689b7368353a430dbffa1cf44f9b5949 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:21.165268 2999 ts_tablet_manager.cc:1888] T 689b7368353a430dbffa1cf44f9b5949 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 20:46:21.165331 2999 log.cc:1196] T 689b7368353a430dbffa1cf44f9b5949 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/689b7368353a430dbffa1cf44f9b5949
I20240502 20:46:21.165782 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet 689b7368353a430dbffa1cf44f9b5949 (table rebalance_test_table_1 [id=8011864bab7b4506a89a8bbb019d244f]) successfully deleted
I20240502 20:46:21.239326 29036 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_24"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:21.248111 31475 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:21.250419 31606 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:21.254498 31475 tablet_service.cc:1467] Processing CreateTablet for tablet d9ebeb5b36674aaf99f35180a61f9cfc (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=7ce5df03ed1d4bb0ab1071f62c1395b9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:21.254783 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9ebeb5b36674aaf99f35180a61f9cfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:21.255339 29163 tablet_service.cc:1467] Processing CreateTablet for tablet 7a8b04cb606f423f8dd1d489d47408d2 (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=7ce5df03ed1d4bb0ab1071f62c1395b9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:21.255353 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 7a8b04cb606f423f8dd1d489d47408d2 (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=7ce5df03ed1d4bb0ab1071f62c1395b9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:21.255584 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a8b04cb606f423f8dd1d489d47408d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:21.255628 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a8b04cb606f423f8dd1d489d47408d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:21.257223 2645 tablet_bootstrap.cc:492] T d9ebeb5b36674aaf99f35180a61f9cfc P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:21.257776 2736 tablet_bootstrap.cc:492] T 7a8b04cb606f423f8dd1d489d47408d2 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:21.258119 2645 tablet_bootstrap.cc:654] T d9ebeb5b36674aaf99f35180a61f9cfc P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:21.258065 29424 tablet_service.cc:1467] Processing CreateTablet for tablet d9ebeb5b36674aaf99f35180a61f9cfc (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=7ce5df03ed1d4bb0ab1071f62c1395b9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:21.258335 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9ebeb5b36674aaf99f35180a61f9cfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:21.258996 31605 tablet_service.cc:1467] Processing CreateTablet for tablet b15554a988034cc7ab18ac2330f1a356 (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=7ce5df03ed1d4bb0ab1071f62c1395b9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:21.259246 31605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b15554a988034cc7ab18ac2330f1a356. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:21.259429 2736 tablet_bootstrap.cc:654] T 7a8b04cb606f423f8dd1d489d47408d2 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:21.260936 29037 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:21.261073 31737 tablet_service.cc:1467] Processing CreateTablet for tablet b15554a988034cc7ab18ac2330f1a356 (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=7ce5df03ed1d4bb0ab1071f62c1395b9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:21.261349 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b15554a988034cc7ab18ac2330f1a356. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:21.261581 29557 tablet_service.cc:1467] Processing CreateTablet for tablet b15554a988034cc7ab18ac2330f1a356 (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=7ce5df03ed1d4bb0ab1071f62c1395b9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:21.261826 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b15554a988034cc7ab18ac2330f1a356. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:21.261909 2736 tablet_bootstrap.cc:492] T 7a8b04cb606f423f8dd1d489d47408d2 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:21.261989 2736 ts_tablet_manager.cc:1367] T 7a8b04cb606f423f8dd1d489d47408d2 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:21.262423 2736 raft_consensus.cc:348] T 7a8b04cb606f423f8dd1d489d47408d2 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:21.262558 2736 raft_consensus.cc:374] T 7a8b04cb606f423f8dd1d489d47408d2 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:21.262591 2736 raft_consensus.cc:729] T 7a8b04cb606f423f8dd1d489d47408d2 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:21.262704 2736 consensus_queue.cc:260] T 7a8b04cb606f423f8dd1d489d47408d2 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:21.262957 2736 ts_tablet_manager.cc:1398] T 7a8b04cb606f423f8dd1d489d47408d2 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:21.263635 2976 tablet_bootstrap.cc:492] T b15554a988034cc7ab18ac2330f1a356 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:21.264542 2976 tablet_bootstrap.cc:654] T b15554a988034cc7ab18ac2330f1a356 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:21.265229 31606 tablet_service.cc:1467] Processing CreateTablet for tablet 7a8b04cb606f423f8dd1d489d47408d2 (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=7ce5df03ed1d4bb0ab1071f62c1395b9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:21.265483 31606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a8b04cb606f423f8dd1d489d47408d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:21.266080 3145 tablet_bootstrap.cc:492] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:21.267462 3145 tablet_bootstrap.cc:654] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:21.268556 3145 tablet_bootstrap.cc:492] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:21.268643 3145 ts_tablet_manager.cc:1367] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:21.269052 3145 raft_consensus.cc:348] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:21.269161 3145 raft_consensus.cc:374] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:21.269187 3145 raft_consensus.cc:729] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:21.269307 3145 consensus_queue.cc:260] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:21.269562 3145 ts_tablet_manager.cc:1398] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:21.271000 31605 tablet_service.cc:1467] Processing CreateTablet for tablet d9ebeb5b36674aaf99f35180a61f9cfc (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=7ce5df03ed1d4bb0ab1071f62c1395b9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:21.271256 31605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9ebeb5b36674aaf99f35180a61f9cfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:21.273780 29557 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:21.276822 29424 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:21.277213 3146 tablet_bootstrap.cc:492] T b15554a988034cc7ab18ac2330f1a356 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:21.278045 3146 tablet_bootstrap.cc:654] T b15554a988034cc7ab18ac2330f1a356 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:21.278885 2976 tablet_bootstrap.cc:492] T b15554a988034cc7ab18ac2330f1a356 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:21.278968 2976 ts_tablet_manager.cc:1367] T b15554a988034cc7ab18ac2330f1a356 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 20:46:21.279412 2976 raft_consensus.cc:348] T b15554a988034cc7ab18ac2330f1a356 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:21.279522 2976 raft_consensus.cc:374] T b15554a988034cc7ab18ac2330f1a356 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:21.279547 2976 raft_consensus.cc:729] T b15554a988034cc7ab18ac2330f1a356 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:21.279645 2976 consensus_queue.cc:260] T b15554a988034cc7ab18ac2330f1a356 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:21.279913 2976 ts_tablet_manager.cc:1398] T b15554a988034cc7ab18ac2330f1a356 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:21.281785 2645 tablet_bootstrap.cc:492] T d9ebeb5b36674aaf99f35180a61f9cfc P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:21.281872 2645 ts_tablet_manager.cc:1367] T d9ebeb5b36674aaf99f35180a61f9cfc P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20240502 20:46:21.282264 2645 raft_consensus.cc:348] T d9ebeb5b36674aaf99f35180a61f9cfc P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:21.282361 2645 raft_consensus.cc:374] T d9ebeb5b36674aaf99f35180a61f9cfc P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:21.282384 2645 raft_consensus.cc:729] T d9ebeb5b36674aaf99f35180a61f9cfc P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:21.282518 2645 consensus_queue.cc:260] T d9ebeb5b36674aaf99f35180a61f9cfc P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:21.282770 2645 ts_tablet_manager.cc:1398] T d9ebeb5b36674aaf99f35180a61f9cfc P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:21.283886 3082 tablet.cc:1722] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:21.284158 3082 tablet.cc:1620] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:21.284245 3083 tablet.cc:1722] T e6c4249d94864dd2bdf4df4947d07eb4 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:21.284533 3083 tablet.cc:1620] T e6c4249d94864dd2bdf4df4947d07eb4 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:21.290982 29038 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=e73e91b41c7348b5bf1cdb369d78588c] alter complete (version 2)
I20240502 20:46:21.291194 3084 tablet.cc:1722] T e6c4249d94864dd2bdf4df4947d07eb4 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:21.291487 3084 tablet.cc:1620] T e6c4249d94864dd2bdf4df4947d07eb4 P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:21.296319 29163 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:21.302361 2915 raft_consensus.cc:482] T b15554a988034cc7ab18ac2330f1a356 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:21.302451 2915 raft_consensus.cc:504] T b15554a988034cc7ab18ac2330f1a356 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:21.302824 2915 leader_election.cc:288] T b15554a988034cc7ab18ac2330f1a356 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:21.303259 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15554a988034cc7ab18ac2330f1a356" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:21.303260 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15554a988034cc7ab18ac2330f1a356" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
W20240502 20:46:21.303622 31691 leader_election.cc:341] T b15554a988034cc7ab18ac2330f1a356 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): Illegal state: must be running to vote when last-logged opid is not known
W20240502 20:46:21.303797 31691 leader_election.cc:341] T b15554a988034cc7ab18ac2330f1a356 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Illegal state: must be running to vote when last-logged opid is not known
I20240502 20:46:21.303848 31691 leader_election.cc:302] T b15554a988034cc7ab18ac2330f1a356 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7ab6b9053664ee5aa71c7978e26fe6a; no voters: 315915efcf2b413ca7f9d874338e1b2d, db17f26a73f3447b92aa9a947e3a7057
I20240502 20:46:21.305126 2915 raft_consensus.cc:2726] T b15554a988034cc7ab18ac2330f1a356 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 20:46:21.309754 3142 tablet_bootstrap.cc:492] T 7a8b04cb606f423f8dd1d489d47408d2 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:21.309754 31867 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:21.310784 3147 tablet_bootstrap.cc:492] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:21.313551 31737 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:21.313618 29293 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:21.321763 1321 raft_consensus.cc:482] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:21.321864 1321 raft_consensus.cc:504] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:21.322250 1321 leader_election.cc:288] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:21.322693 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9ebeb5b36674aaf99f35180a61f9cfc" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
W20240502 20:46:21.323064 29380 leader_election.cc:341] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Illegal state: must be running to vote when last-logged opid is not known
I20240502 20:46:21.323488 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9ebeb5b36674aaf99f35180a61f9cfc" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:21.323629 31495 raft_consensus.cc:2454] T d9ebeb5b36674aaf99f35180a61f9cfc P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:21.323984 29381 leader_election.cc:302] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, dbcdc6ac635b44b185c3eba63f66e9d6; no voters: 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:21.324201 1321 raft_consensus.cc:2781] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:21.324254 1321 raft_consensus.cc:482] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:21.324293 1321 raft_consensus.cc:3037] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:21.325361 1321 raft_consensus.cc:504] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:21.325695 1321 leader_election.cc:288] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:21.326117 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9ebeb5b36674aaf99f35180a61f9cfc" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
W20240502 20:46:21.326424 29380 leader_election.cc:341] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Illegal state: must be running to vote when last-logged opid is not known
I20240502 20:46:21.327117 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9ebeb5b36674aaf99f35180a61f9cfc" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:21.327229 31495 raft_consensus.cc:3037] T d9ebeb5b36674aaf99f35180a61f9cfc P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:21.328208 31495 raft_consensus.cc:2454] T d9ebeb5b36674aaf99f35180a61f9cfc P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:21.328537 29381 leader_election.cc:302] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, dbcdc6ac635b44b185c3eba63f66e9d6; no voters: 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:21.328727 1321 raft_consensus.cc:2781] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:21.328822 1321 raft_consensus.cc:686] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:21.328938 1321 consensus_queue.cc:237] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:21.330857 29036 catalog_manager.cc:5484] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:21.340035 3147 tablet_bootstrap.cc:654] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:21.341542 3142 tablet_bootstrap.cc:654] T 7a8b04cb606f423f8dd1d489d47408d2 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:21.345283 3147 tablet_bootstrap.cc:492] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:21.345382 3147 ts_tablet_manager.cc:1367] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.035s user 0.002s sys 0.000s
I20240502 20:46:21.345780 3147 raft_consensus.cc:348] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:21.347107 3142 tablet_bootstrap.cc:492] T 7a8b04cb606f423f8dd1d489d47408d2 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:21.347211 3142 ts_tablet_manager.cc:1367] T 7a8b04cb606f423f8dd1d489d47408d2 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.038s user 0.002s sys 0.000s
I20240502 20:46:21.347694 3142 raft_consensus.cc:348] T 7a8b04cb606f423f8dd1d489d47408d2 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:21.347810 3142 raft_consensus.cc:374] T 7a8b04cb606f423f8dd1d489d47408d2 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:21.347883 3142 raft_consensus.cc:729] T 7a8b04cb606f423f8dd1d489d47408d2 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:21.348016 3142 consensus_queue.cc:260] T 7a8b04cb606f423f8dd1d489d47408d2 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:21.348294 3142 ts_tablet_manager.cc:1398] T 7a8b04cb606f423f8dd1d489d47408d2 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:21.349432 3147 raft_consensus.cc:374] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:21.349545 3147 raft_consensus.cc:729] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:21.349706 3147 consensus_queue.cc:260] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:21.349956 3147 ts_tablet_manager.cc:1398] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 20:46:21.352627 3146 tablet_bootstrap.cc:492] T b15554a988034cc7ab18ac2330f1a356 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:21.352737 3146 ts_tablet_manager.cc:1367] T b15554a988034cc7ab18ac2330f1a356 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.076s user 0.002s sys 0.000s
I20240502 20:46:21.353200 3146 raft_consensus.cc:348] T b15554a988034cc7ab18ac2330f1a356 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:21.353286 3146 raft_consensus.cc:374] T b15554a988034cc7ab18ac2330f1a356 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:21.353307 3146 raft_consensus.cc:729] T b15554a988034cc7ab18ac2330f1a356 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:21.353408 3146 consensus_queue.cc:260] T b15554a988034cc7ab18ac2330f1a356 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:21.353670 3146 ts_tablet_manager.cc:1398] T b15554a988034cc7ab18ac2330f1a356 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:21.353971 3146 tablet_bootstrap.cc:492] T 7a8b04cb606f423f8dd1d489d47408d2 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:21.354977 3146 tablet_bootstrap.cc:654] T 7a8b04cb606f423f8dd1d489d47408d2 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:21.367539 3146 tablet_bootstrap.cc:492] T 7a8b04cb606f423f8dd1d489d47408d2 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:21.367697 3146 ts_tablet_manager.cc:1367] T 7a8b04cb606f423f8dd1d489d47408d2 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.002s
I20240502 20:46:21.368259 3146 raft_consensus.cc:348] T 7a8b04cb606f423f8dd1d489d47408d2 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:21.368458 3146 raft_consensus.cc:374] T 7a8b04cb606f423f8dd1d489d47408d2 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:21.368537 3146 raft_consensus.cc:729] T 7a8b04cb606f423f8dd1d489d47408d2 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:21.368734 3146 consensus_queue.cc:260] T 7a8b04cb606f423f8dd1d489d47408d2 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:21.371582 3146 ts_tablet_manager.cc:1398] T 7a8b04cb606f423f8dd1d489d47408d2 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20240502 20:46:21.372306 3146 tablet_bootstrap.cc:492] T d9ebeb5b36674aaf99f35180a61f9cfc P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:21.373459 3146 tablet_bootstrap.cc:654] T d9ebeb5b36674aaf99f35180a61f9cfc P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:21.375020 3146 tablet_bootstrap.cc:492] T d9ebeb5b36674aaf99f35180a61f9cfc P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:21.375095 3146 ts_tablet_manager.cc:1367] T d9ebeb5b36674aaf99f35180a61f9cfc P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 20:46:21.375491 3146 raft_consensus.cc:348] T d9ebeb5b36674aaf99f35180a61f9cfc P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:21.375572 3146 raft_consensus.cc:374] T d9ebeb5b36674aaf99f35180a61f9cfc P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:21.375594 3146 raft_consensus.cc:729] T d9ebeb5b36674aaf99f35180a61f9cfc P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:21.375700 3146 consensus_queue.cc:260] T d9ebeb5b36674aaf99f35180a61f9cfc P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:21.375975 3146 ts_tablet_manager.cc:1398] T d9ebeb5b36674aaf99f35180a61f9cfc P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:21.391436 29182 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8c21a6762eaa4ba4ac1c91a42fb37849"
dest_uuid: "7796ea3fba294d17a34386873969431d"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:59776
I20240502 20:46:21.391613 29182 raft_consensus.cc:595] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Received request to transfer leadership
I20240502 20:46:21.397042 29036 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:21.405979 3082 tablet.cc:1722] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:21.406553 3084 tablet.cc:1722] T e6c4249d94864dd2bdf4df4947d07eb4 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:21.406834 3083 tablet.cc:1722] T e6c4249d94864dd2bdf4df4947d07eb4 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:21.412792 29039 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=e73e91b41c7348b5bf1cdb369d78588c] alter complete (version 3)
I20240502 20:46:21.467206 2916 raft_consensus.cc:482] T b15554a988034cc7ab18ac2330f1a356 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:21.467343 2916 raft_consensus.cc:504] T b15554a988034cc7ab18ac2330f1a356 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:21.467825 2916 leader_election.cc:288] T b15554a988034cc7ab18ac2330f1a356 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:21.468202 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15554a988034cc7ab18ac2330f1a356" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:21.468254 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15554a988034cc7ab18ac2330f1a356" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:21.468348 29577 raft_consensus.cc:2454] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:21.468389 31756 raft_consensus.cc:2454] T b15554a988034cc7ab18ac2330f1a356 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:21.468721 31559 leader_election.cc:302] T b15554a988034cc7ab18ac2330f1a356 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:21.468986 2916 raft_consensus.cc:2781] T b15554a988034cc7ab18ac2330f1a356 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:21.469043 2916 raft_consensus.cc:482] T b15554a988034cc7ab18ac2330f1a356 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:21.469077 2916 raft_consensus.cc:3037] T b15554a988034cc7ab18ac2330f1a356 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:21.470207 2916 raft_consensus.cc:504] T b15554a988034cc7ab18ac2330f1a356 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:21.470558 2916 leader_election.cc:288] T b15554a988034cc7ab18ac2330f1a356 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:21.471412 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15554a988034cc7ab18ac2330f1a356" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:21.471541 29577 raft_consensus.cc:3037] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:21.472571 29577 raft_consensus.cc:2454] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:21.472939 31560 leader_election.cc:302] T b15554a988034cc7ab18ac2330f1a356 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:21.473322 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15554a988034cc7ab18ac2330f1a356" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:21.473438 31756 raft_consensus.cc:3037] T b15554a988034cc7ab18ac2330f1a356 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:21.474421 31756 raft_consensus.cc:2454] T b15554a988034cc7ab18ac2330f1a356 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:21.474969 2916 raft_consensus.cc:2781] T b15554a988034cc7ab18ac2330f1a356 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:21.475054 2916 raft_consensus.cc:686] T b15554a988034cc7ab18ac2330f1a356 P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Becoming Leader. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Running, Role: LEADER
I20240502 20:46:21.475185 2916 consensus_queue.cc:237] T b15554a988034cc7ab18ac2330f1a356 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:21.477732 29042 catalog_manager.cc:5484] T b15554a988034cc7ab18ac2330f1a356 P 315915efcf2b413ca7f9d874338e1b2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70). New cstate: current_term: 1 leader_uuid: "315915efcf2b413ca7f9d874338e1b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:21.487684 2882 raft_consensus.cc:482] T 7a8b04cb606f423f8dd1d489d47408d2 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:21.487787 2882 raft_consensus.cc:504] T 7a8b04cb606f423f8dd1d489d47408d2 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:21.488166 2882 leader_election.cc:288] T 7a8b04cb606f423f8dd1d489d47408d2 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:21.488915 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a8b04cb606f423f8dd1d489d47408d2" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:21.489003 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a8b04cb606f423f8dd1d489d47408d2" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:21.489054 29182 raft_consensus.cc:2454] T 7a8b04cb606f423f8dd1d489d47408d2 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:21.489116 31626 raft_consensus.cc:2454] T 7a8b04cb606f423f8dd1d489d47408d2 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:21.489403 31824 leader_election.cc:302] T 7a8b04cb606f423f8dd1d489d47408d2 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:21.489640 2882 raft_consensus.cc:2781] T 7a8b04cb606f423f8dd1d489d47408d2 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:21.489702 2882 raft_consensus.cc:482] T 7a8b04cb606f423f8dd1d489d47408d2 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:21.489748 2882 raft_consensus.cc:3037] T 7a8b04cb606f423f8dd1d489d47408d2 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:21.490837 2882 raft_consensus.cc:504] T 7a8b04cb606f423f8dd1d489d47408d2 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:21.491199 2882 leader_election.cc:288] T 7a8b04cb606f423f8dd1d489d47408d2 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:21.491212 2144 raft_consensus.cc:979] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d: : Instructing follower db17f26a73f3447b92aa9a947e3a7057 to start an election
I20240502 20:46:21.491302 2144 raft_consensus.cc:1067] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Signalling peer db17f26a73f3447b92aa9a947e3a7057 to start an election
I20240502 20:46:21.491633 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a8b04cb606f423f8dd1d489d47408d2" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:21.491748 29182 raft_consensus.cc:3037] T 7a8b04cb606f423f8dd1d489d47408d2 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:21.491994 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a8b04cb606f423f8dd1d489d47408d2" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:21.492090 31626 raft_consensus.cc:3037] T 7a8b04cb606f423f8dd1d489d47408d2 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:21.492779 29182 raft_consensus.cc:2454] T 7a8b04cb606f423f8dd1d489d47408d2 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:21.492903 31626 raft_consensus.cc:2454] T 7a8b04cb606f423f8dd1d489d47408d2 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:21.493101 31824 leader_election.cc:302] T 7a8b04cb606f423f8dd1d489d47408d2 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:21.493330 29577 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8c21a6762eaa4ba4ac1c91a42fb37849"
dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
from {username='slave'} at 127.22.45.65:49965
I20240502 20:46:21.493369 2882 raft_consensus.cc:2781] T 7a8b04cb606f423f8dd1d489d47408d2 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:21.493475 29577 raft_consensus.cc:482] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:21.493537 29577 raft_consensus.cc:3037] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:21.493530 2882 raft_consensus.cc:686] T 7a8b04cb606f423f8dd1d489d47408d2 P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Becoming Leader. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Running, Role: LEADER
I20240502 20:46:21.493674 2882 consensus_queue.cc:237] T 7a8b04cb606f423f8dd1d489d47408d2 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:21.494412 29577 raft_consensus.cc:504] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:21.495870 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c21a6762eaa4ba4ac1c91a42fb37849" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:21.495967 29182 raft_consensus.cc:3032] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:21.496002 29182 raft_consensus.cc:729] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:21.496070 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c21a6762eaa4ba4ac1c91a42fb37849" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:21.496142 29182 consensus_queue.cc:260] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:21.496193 29444 raft_consensus.cc:3037] T 8c21a6762eaa4ba4ac1c91a42fb37849 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:21.496315 29182 raft_consensus.cc:3037] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:21.496438 29041 catalog_manager.cc:5484] T 7a8b04cb606f423f8dd1d489d47408d2 P e6b60803d0764d228c863e2da65e3691 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6b60803d0764d228c863e2da65e3691 (127.22.45.72). New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:21.497138 29182 raft_consensus.cc:2454] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 2.
I20240502 20:46:21.497151 29444 raft_consensus.cc:2454] T 8c21a6762eaa4ba4ac1c91a42fb37849 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 2.
I20240502 20:46:21.497377 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c21a6762eaa4ba4ac1c91a42fb37849" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:21.497473 31888 raft_consensus.cc:3037] T 8c21a6762eaa4ba4ac1c91a42fb37849 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:21.498255 31888 raft_consensus.cc:2454] T 8c21a6762eaa4ba4ac1c91a42fb37849 P e6b60803d0764d228c863e2da65e3691 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 2.
I20240502 20:46:21.498605 29511 leader_election.cc:302] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, dbcdc6ac635b44b185c3eba63f66e9d6, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:21.498775 2998 raft_consensus.cc:2781] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:21.498845 2998 raft_consensus.cc:686] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057 [term 2 LEADER]: Becoming Leader. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Running, Role: LEADER
I20240502 20:46:21.498948 2998 consensus_queue.cc:237] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:21.500747 29577 leader_election.cc:288] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 2 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:21.500893 29041 catalog_manager.cc:5484] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: term changed from 1 to 2, leader changed from 7796ea3fba294d17a34386873969431d (127.22.45.65) to db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68). New cstate: current_term: 2 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:21.516165 29293 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:21.517158 29041 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:21.517306 29041 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:21.518187 31475 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:21.520673 29041 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet e6c4249d94864dd2bdf4df4947d07eb4
I20240502 20:46:21.521792 31737 tablet_service.cc:1514] Processing DeleteTablet for tablet e6c4249d94864dd2bdf4df4947d07eb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:21 UTC) from {username='slave'} at 127.0.0.1:33864
I20240502 20:46:21.522099 31867 tablet_service.cc:1514] Processing DeleteTablet for tablet e6c4249d94864dd2bdf4df4947d07eb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:21 UTC) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:21.522567 31475 tablet_service.cc:1514] Processing DeleteTablet for tablet e6c4249d94864dd2bdf4df4947d07eb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:21 UTC) from {username='slave'} at 127.0.0.1:40276
I20240502 20:46:21.526569 3215 tablet_replica.cc:330] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe: stopping tablet replica
I20240502 20:46:21.526744 3214 tablet_replica.cc:330] T e6c4249d94864dd2bdf4df4947d07eb4 P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:21.527168 3215 raft_consensus.cc:2229] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:21.527290 3214 raft_consensus.cc:2229] T e6c4249d94864dd2bdf4df4947d07eb4 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:21.527475 3214 raft_consensus.cc:2258] T e6c4249d94864dd2bdf4df4947d07eb4 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:21.527727 3215 raft_consensus.cc:2258] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:21.530298 3215 ts_tablet_manager.cc:1875] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:21.531471 3214 ts_tablet_manager.cc:1875] T e6c4249d94864dd2bdf4df4947d07eb4 P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:21.531488 29163 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:21.534255 31737 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:21.541849 3213 tablet_replica.cc:330] T e6c4249d94864dd2bdf4df4947d07eb4 P e7ab6b9053664ee5aa71c7978e26fe6a: stopping tablet replica
I20240502 20:46:21.542037 3213 raft_consensus.cc:2229] T e6c4249d94864dd2bdf4df4947d07eb4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:21.542181 3213 raft_consensus.cc:2258] T e6c4249d94864dd2bdf4df4947d07eb4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:21.542821 3213 ts_tablet_manager.cc:1875] T e6c4249d94864dd2bdf4df4947d07eb4 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:21.544880 31605 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:21.545248 3213 ts_tablet_manager.cc:1888] T e6c4249d94864dd2bdf4df4947d07eb4 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:21.545378 3213 log.cc:1196] T e6c4249d94864dd2bdf4df4947d07eb4 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/wals/e6c4249d94864dd2bdf4df4947d07eb4
I20240502 20:46:21.545709 3213 ts_tablet_manager.cc:1909] T e6c4249d94864dd2bdf4df4947d07eb4 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting consensus metadata
I20240502 20:46:21.546434 29026 catalog_manager.cc:4830] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): tablet e6c4249d94864dd2bdf4df4947d07eb4 (table rebalancer_dynamic_table_new_name [id=e73e91b41c7348b5bf1cdb369d78588c]) successfully deleted
I20240502 20:46:21.572578 29557 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:21.573019 31867 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:21.575562 3215 ts_tablet_manager.cc:1888] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:21.577697 3214 ts_tablet_manager.cc:1888] T e6c4249d94864dd2bdf4df4947d07eb4 P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:21.577780 3214 log.cc:1196] T e6c4249d94864dd2bdf4df4947d07eb4 P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/e6c4249d94864dd2bdf4df4947d07eb4
I20240502 20:46:21.577922 3215 log.cc:1196] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/wals/e6c4249d94864dd2bdf4df4947d07eb4
I20240502 20:46:21.578471 3215 ts_tablet_manager.cc:1909] T e6c4249d94864dd2bdf4df4947d07eb4 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting consensus metadata
I20240502 20:46:21.578684 3214 ts_tablet_manager.cc:1909] T e6c4249d94864dd2bdf4df4947d07eb4 P e6b60803d0764d228c863e2da65e3691: Deleting consensus metadata
I20240502 20:46:21.580145 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet e6c4249d94864dd2bdf4df4947d07eb4 (table rebalancer_dynamic_table_new_name [id=e73e91b41c7348b5bf1cdb369d78588c]) successfully deleted
I20240502 20:46:21.580157 29028 catalog_manager.cc:4830] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): tablet e6c4249d94864dd2bdf4df4947d07eb4 (table rebalancer_dynamic_table_new_name [id=e73e91b41c7348b5bf1cdb369d78588c]) successfully deleted
I20240502 20:46:21.593884 29424 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:21.623348 29041 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:21.629774 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 514c4b61a86745f793a2375d09184475 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=be2a9e7ea3de486180ad95784329fbdb]), partition=
I20240502 20:46:21.629864 31605 tablet_service.cc:1467] Processing CreateTablet for tablet 514c4b61a86745f793a2375d09184475 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=be2a9e7ea3de486180ad95784329fbdb]), partition=
I20240502 20:46:21.629887 31737 tablet_service.cc:1467] Processing CreateTablet for tablet 514c4b61a86745f793a2375d09184475 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=be2a9e7ea3de486180ad95784329fbdb]), partition=
I20240502 20:46:21.630051 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 514c4b61a86745f793a2375d09184475. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:21.630100 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 514c4b61a86745f793a2375d09184475. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:21.630101 31605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 514c4b61a86745f793a2375d09184475. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:21.631750 3146 tablet_bootstrap.cc:492] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:21.632109 2736 tablet_bootstrap.cc:492] T 514c4b61a86745f793a2375d09184475 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:21.632690 3146 tablet_bootstrap.cc:654] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:21.633005 2736 tablet_bootstrap.cc:654] T 514c4b61a86745f793a2375d09184475 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:21.633579 2976 tablet_bootstrap.cc:492] T 514c4b61a86745f793a2375d09184475 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:21.634373 3146 tablet_bootstrap.cc:492] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:21.634447 2976 tablet_bootstrap.cc:654] T 514c4b61a86745f793a2375d09184475 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:21.634454 3146 ts_tablet_manager.cc:1367] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 20:46:21.634965 3146 raft_consensus.cc:348] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:21.635071 3146 raft_consensus.cc:374] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:21.635102 3146 raft_consensus.cc:729] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:21.635222 3146 consensus_queue.cc:260] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:21.635483 3146 ts_tablet_manager.cc:1398] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:21.636301 2736 tablet_bootstrap.cc:492] T 514c4b61a86745f793a2375d09184475 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:21.636373 2736 ts_tablet_manager.cc:1367] T 514c4b61a86745f793a2375d09184475 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:21.636801 2736 raft_consensus.cc:348] T 514c4b61a86745f793a2375d09184475 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:21.636898 2736 raft_consensus.cc:374] T 514c4b61a86745f793a2375d09184475 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:21.636927 2736 raft_consensus.cc:729] T 514c4b61a86745f793a2375d09184475 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:21.637042 2736 consensus_queue.cc:260] T 514c4b61a86745f793a2375d09184475 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:21.637323 2736 ts_tablet_manager.cc:1398] T 514c4b61a86745f793a2375d09184475 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:21.642715 2976 tablet_bootstrap.cc:492] T 514c4b61a86745f793a2375d09184475 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:21.642798 2976 ts_tablet_manager.cc:1367] T 514c4b61a86745f793a2375d09184475 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.002s
I20240502 20:46:21.643200 2976 raft_consensus.cc:348] T 514c4b61a86745f793a2375d09184475 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:21.643288 2976 raft_consensus.cc:374] T 514c4b61a86745f793a2375d09184475 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:21.643311 2976 raft_consensus.cc:729] T 514c4b61a86745f793a2375d09184475 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:21.643421 2976 consensus_queue.cc:260] T 514c4b61a86745f793a2375d09184475 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:21.643713 2976 ts_tablet_manager.cc:1398] T 514c4b61a86745f793a2375d09184475 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:21.753213 2916 raft_consensus.cc:482] T d9ebeb5b36674aaf99f35180a61f9cfc P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:21.753327 2916 raft_consensus.cc:504] T d9ebeb5b36674aaf99f35180a61f9cfc P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:21.753845 2916 leader_election.cc:288] T d9ebeb5b36674aaf99f35180a61f9cfc P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:21.754187 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9ebeb5b36674aaf99f35180a61f9cfc" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:21.754233 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9ebeb5b36674aaf99f35180a61f9cfc" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:21.754333 31495 raft_consensus.cc:2379] T d9ebeb5b36674aaf99f35180a61f9cfc P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 315915efcf2b413ca7f9d874338e1b2d in current term 1: Already voted for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in this term.
I20240502 20:46:21.754642 31561 leader_election.cc:302] T d9ebeb5b36674aaf99f35180a61f9cfc P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d; no voters: 2a9c1af8cf814030bde56b67ca5c2fbe, dbcdc6ac635b44b185c3eba63f66e9d6
I20240502 20:46:21.754812 2916 raft_consensus.cc:3037] T d9ebeb5b36674aaf99f35180a61f9cfc P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:21.755801 2916 raft_consensus.cc:2726] T d9ebeb5b36674aaf99f35180a61f9cfc P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 20:46:21.800527 3224 consensus_queue.cc:1035] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:21.804435 3224 consensus_queue.cc:1035] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:21.928148 31888 raft_consensus.cc:1261] T 8c21a6762eaa4ba4ac1c91a42fb37849 P e6b60803d0764d228c863e2da65e3691 [term 2 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:21.928560 2998 consensus_queue.cc:1035] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:21.955734 2916 consensus_queue.cc:1035] T b15554a988034cc7ab18ac2330f1a356 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:21.958640 2916 consensus_queue.cc:1035] T b15554a988034cc7ab18ac2330f1a356 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:21.975373 2916 raft_consensus.cc:482] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:21.975474 2916 raft_consensus.cc:504] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:21.975831 2916 leader_election.cc:288] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:21.976249 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514c4b61a86745f793a2375d09184475" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:21.976279 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514c4b61a86745f793a2375d09184475" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:21.976387 31756 raft_consensus.cc:2454] T 514c4b61a86745f793a2375d09184475 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:21.976387 31888 raft_consensus.cc:2454] T 514c4b61a86745f793a2375d09184475 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:21.976613 31560 leader_election.cc:302] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:21.976886 2916 raft_consensus.cc:2781] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:21.976946 2916 raft_consensus.cc:482] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:21.976980 2916 raft_consensus.cc:3037] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:21.977957 2916 raft_consensus.cc:504] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:21.978302 2916 leader_election.cc:288] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Requested vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:21.978679 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514c4b61a86745f793a2375d09184475" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:21.978817 31888 raft_consensus.cc:3037] T 514c4b61a86745f793a2375d09184475 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:21.978811 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514c4b61a86745f793a2375d09184475" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:21.978932 31756 raft_consensus.cc:3037] T 514c4b61a86745f793a2375d09184475 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:21.979669 31888 raft_consensus.cc:2454] T 514c4b61a86745f793a2375d09184475 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:21.979725 31756 raft_consensus.cc:2454] T 514c4b61a86745f793a2375d09184475 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:21.979918 31560 leader_election.cc:302] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:21.980075 2916 raft_consensus.cc:2781] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:21.980162 2916 raft_consensus.cc:686] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Becoming Leader. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Running, Role: LEADER
I20240502 20:46:21.980294 2916 consensus_queue.cc:237] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:21.982097 29036 catalog_manager.cc:5484] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70). New cstate: current_term: 1 leader_uuid: "315915efcf2b413ca7f9d874338e1b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:22.030350 2882 consensus_queue.cc:1035] T 7a8b04cb606f423f8dd1d489d47408d2 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:22.031811 29163 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:22.032368 31605 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:22.034511 29293 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:22.037580 29557 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:22.039541 29182 raft_consensus.cc:1261] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [term 2 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:22.040918 2882 consensus_queue.cc:1035] T 7a8b04cb606f423f8dd1d489d47408d2 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:22.041303 2998 consensus_queue.cc:1035] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:22.044049 31475 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:22.044379 29033 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_26"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:22.046296 29424 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:22.054602 31867 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:22.058307 31737 tablet_service.cc:1467] Processing CreateTablet for tablet 2227f1ae58a044ea955cffa8d8d8a3f2 (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=288d696f58ef4fb08ffcc029da0478db]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:22.058498 31475 tablet_service.cc:1467] Processing CreateTablet for tablet 2227f1ae58a044ea955cffa8d8d8a3f2 (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=288d696f58ef4fb08ffcc029da0478db]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:22.058636 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2227f1ae58a044ea955cffa8d8d8a3f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:22.058775 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2227f1ae58a044ea955cffa8d8d8a3f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:22.058907 31474 tablet_service.cc:1467] Processing CreateTablet for tablet d6b1c1798aaf4f1fb3d6ef52503e08be (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=288d696f58ef4fb08ffcc029da0478db]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:22.059114 31474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6b1c1798aaf4f1fb3d6ef52503e08be. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:22.061097 2976 tablet_bootstrap.cc:492] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:22.061995 2976 tablet_bootstrap.cc:654] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:22.063082 31737 tablet_service.cc:1467] Processing CreateTablet for tablet d6b1c1798aaf4f1fb3d6ef52503e08be (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=288d696f58ef4fb08ffcc029da0478db]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:22.063973 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6b1c1798aaf4f1fb3d6ef52503e08be. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:22.065027 31474 tablet_service.cc:1467] Processing CreateTablet for tablet 6042767df75143fd9f873d51fefbc9f7 (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=288d696f58ef4fb08ffcc029da0478db]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:22.065251 31474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6042767df75143fd9f873d51fefbc9f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:22.066691 31605 tablet_service.cc:1467] Processing CreateTablet for tablet d6b1c1798aaf4f1fb3d6ef52503e08be (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=288d696f58ef4fb08ffcc029da0478db]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:22.066964 31605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6b1c1798aaf4f1fb3d6ef52503e08be. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:22.068734 3291 tablet_bootstrap.cc:492] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:22.069049 3146 tablet_bootstrap.cc:492] T d6b1c1798aaf4f1fb3d6ef52503e08be P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:22.069653 3291 tablet_bootstrap.cc:654] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:22.069891 3146 tablet_bootstrap.cc:654] T d6b1c1798aaf4f1fb3d6ef52503e08be P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:22.070745 3291 tablet_bootstrap.cc:492] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:22.070832 3291 ts_tablet_manager.cc:1367] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:22.070883 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 2227f1ae58a044ea955cffa8d8d8a3f2 (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=288d696f58ef4fb08ffcc029da0478db]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:22.071120 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2227f1ae58a044ea955cffa8d8d8a3f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:22.071269 3291 raft_consensus.cc:348] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:22.071375 3291 raft_consensus.cc:374] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:22.071406 3291 raft_consensus.cc:729] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:22.071678 3291 consensus_queue.cc:260] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:22.072558 29444 raft_consensus.cc:1261] T 8c21a6762eaa4ba4ac1c91a42fb37849 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 2 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:22.072857 3291 ts_tablet_manager.cc:1398] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:22.073068 3291 tablet_bootstrap.cc:492] T d6b1c1798aaf4f1fb3d6ef52503e08be P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:22.073225 2736 tablet_bootstrap.cc:492] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:22.074021 3291 tablet_bootstrap.cc:654] T d6b1c1798aaf4f1fb3d6ef52503e08be P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:22.074062 2736 tablet_bootstrap.cc:654] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:22.074981 3291 tablet_bootstrap.cc:492] T d6b1c1798aaf4f1fb3d6ef52503e08be P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:22.075047 3291 ts_tablet_manager.cc:1367] T d6b1c1798aaf4f1fb3d6ef52503e08be P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 20:46:22.075107 2717 consensus_queue.cc:1035] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:22.075453 3291 raft_consensus.cc:348] T d6b1c1798aaf4f1fb3d6ef52503e08be P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:22.075556 3291 raft_consensus.cc:374] T d6b1c1798aaf4f1fb3d6ef52503e08be P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:22.075585 3291 raft_consensus.cc:729] T d6b1c1798aaf4f1fb3d6ef52503e08be P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:22.075691 3291 consensus_queue.cc:260] T d6b1c1798aaf4f1fb3d6ef52503e08be P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:22.075927 3291 ts_tablet_manager.cc:1398] T d6b1c1798aaf4f1fb3d6ef52503e08be P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:22.076049 3291 tablet_bootstrap.cc:492] T 6042767df75143fd9f873d51fefbc9f7 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:22.076802 3291 tablet_bootstrap.cc:654] T 6042767df75143fd9f873d51fefbc9f7 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:22.085672 31737 tablet_service.cc:1467] Processing CreateTablet for tablet 6042767df75143fd9f873d51fefbc9f7 (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=288d696f58ef4fb08ffcc029da0478db]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:22.085940 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6042767df75143fd9f873d51fefbc9f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:22.088191 3291 tablet_bootstrap.cc:492] T 6042767df75143fd9f873d51fefbc9f7 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:22.088275 3291 ts_tablet_manager.cc:1367] T 6042767df75143fd9f873d51fefbc9f7 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.002s
I20240502 20:46:22.088656 3291 raft_consensus.cc:348] T 6042767df75143fd9f873d51fefbc9f7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:22.088752 3291 raft_consensus.cc:374] T 6042767df75143fd9f873d51fefbc9f7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:22.088775 3291 raft_consensus.cc:729] T 6042767df75143fd9f873d51fefbc9f7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:22.088874 3291 consensus_queue.cc:260] T 6042767df75143fd9f873d51fefbc9f7 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:22.089123 3291 ts_tablet_manager.cc:1398] T 6042767df75143fd9f873d51fefbc9f7 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:22.091105 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 6042767df75143fd9f873d51fefbc9f7 (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=288d696f58ef4fb08ffcc029da0478db]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:22.091418 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6042767df75143fd9f873d51fefbc9f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:22.092916 3146 tablet_bootstrap.cc:492] T d6b1c1798aaf4f1fb3d6ef52503e08be P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:22.093009 3146 ts_tablet_manager.cc:1367] T d6b1c1798aaf4f1fb3d6ef52503e08be P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.000s
I20240502 20:46:22.093670 3146 raft_consensus.cc:348] T d6b1c1798aaf4f1fb3d6ef52503e08be P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:22.093780 3146 raft_consensus.cc:374] T d6b1c1798aaf4f1fb3d6ef52503e08be P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:22.093806 3146 raft_consensus.cc:729] T d6b1c1798aaf4f1fb3d6ef52503e08be P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:22.093907 3146 consensus_queue.cc:260] T d6b1c1798aaf4f1fb3d6ef52503e08be P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:22.094162 3146 ts_tablet_manager.cc:1398] T d6b1c1798aaf4f1fb3d6ef52503e08be P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:22.096801 31737 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:22.100425 29036 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:22.102124 2976 tablet_bootstrap.cc:492] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:22.102229 2976 ts_tablet_manager.cc:1367] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.041s user 0.000s sys 0.002s
I20240502 20:46:22.102710 2976 raft_consensus.cc:348] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:22.102805 2976 raft_consensus.cc:374] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:22.102831 2976 raft_consensus.cc:729] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:22.102932 2976 consensus_queue.cc:260] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:22.103354 2976 ts_tablet_manager.cc:1398] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:22.103521 2976 tablet_bootstrap.cc:492] T d6b1c1798aaf4f1fb3d6ef52503e08be P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:22.104494 2976 tablet_bootstrap.cc:654] T d6b1c1798aaf4f1fb3d6ef52503e08be P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:22.105430 31756 raft_consensus.cc:1261] T 514c4b61a86745f793a2375d09184475 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer 315915efcf2b413ca7f9d874338e1b2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:22.105784 2289 consensus_queue.cc:1035] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:22.106087 2736 tablet_bootstrap.cc:492] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:22.106194 2736 ts_tablet_manager.cc:1367] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.000s
I20240502 20:46:22.106648 2736 raft_consensus.cc:348] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:22.106760 2736 raft_consensus.cc:374] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:22.106791 2736 raft_consensus.cc:729] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:22.106895 2736 consensus_queue.cc:260] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:22.107054 31887 raft_consensus.cc:1261] T 514c4b61a86745f793a2375d09184475 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer 315915efcf2b413ca7f9d874338e1b2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:22.107190 2736 ts_tablet_manager.cc:1398] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:22.107358 2736 tablet_bootstrap.cc:492] T 6042767df75143fd9f873d51fefbc9f7 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:22.108304 2289 consensus_queue.cc:1035] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:22.109831 2736 tablet_bootstrap.cc:654] T 6042767df75143fd9f873d51fefbc9f7 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:22.110208 29036 catalog_manager.cc:5576] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787) does not have the latest schema for tablet 514c4b61a86745f793a2375d09184475 (table rebalancer_dynamic_table [id=be2a9e7ea3de486180ad95784329fbdb]). Expected version 1 got 0
I20240502 20:46:22.111747 29036 catalog_manager.cc:5576] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207) does not have the latest schema for tablet 514c4b61a86745f793a2375d09184475 (table rebalancer_dynamic_table [id=be2a9e7ea3de486180ad95784329fbdb]). Expected version 1 got 0
I20240502 20:46:22.112126 2736 tablet_bootstrap.cc:492] T 6042767df75143fd9f873d51fefbc9f7 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:22.112308 2736 ts_tablet_manager.cc:1367] T 6042767df75143fd9f873d51fefbc9f7 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 20:46:22.112576 3303 tablet.cc:1722] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:22.112951 3303 tablet.cc:1620] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:22.112884 2736 raft_consensus.cc:348] T 6042767df75143fd9f873d51fefbc9f7 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:22.113111 2736 raft_consensus.cc:374] T 6042767df75143fd9f873d51fefbc9f7 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:22.113210 2736 raft_consensus.cc:729] T 6042767df75143fd9f873d51fefbc9f7 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:22.113363 2736 consensus_queue.cc:260] T 6042767df75143fd9f873d51fefbc9f7 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:22.114243 29036 catalog_manager.cc:5576] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207) does not have the latest schema for tablet 514c4b61a86745f793a2375d09184475 (table rebalancer_dynamic_table [id=be2a9e7ea3de486180ad95784329fbdb]). Expected version 1 got 0
I20240502 20:46:22.115036 2736 ts_tablet_manager.cc:1398] T 6042767df75143fd9f873d51fefbc9f7 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:22.115615 3304 tablet.cc:1722] T 514c4b61a86745f793a2375d09184475 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:22.115883 3304 tablet.cc:1620] T 514c4b61a86745f793a2375d09184475 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:22.123744 2976 tablet_bootstrap.cc:492] T d6b1c1798aaf4f1fb3d6ef52503e08be P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:22.123837 2976 ts_tablet_manager.cc:1367] T d6b1c1798aaf4f1fb3d6ef52503e08be P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.001s
I20240502 20:46:22.124267 2976 raft_consensus.cc:348] T d6b1c1798aaf4f1fb3d6ef52503e08be P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:22.124379 2976 raft_consensus.cc:374] T d6b1c1798aaf4f1fb3d6ef52503e08be P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:22.124418 2976 raft_consensus.cc:729] T d6b1c1798aaf4f1fb3d6ef52503e08be P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:22.124519 2976 consensus_queue.cc:260] T d6b1c1798aaf4f1fb3d6ef52503e08be P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:22.124801 2976 ts_tablet_manager.cc:1398] T d6b1c1798aaf4f1fb3d6ef52503e08be P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 20:46:22.124871 29036 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=be2a9e7ea3de486180ad95784329fbdb] alter complete (version 1)
I20240502 20:46:22.124979 2976 tablet_bootstrap.cc:492] T 6042767df75143fd9f873d51fefbc9f7 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:22.124814 3305 tablet.cc:1722] T 514c4b61a86745f793a2375d09184475 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:22.125342 3305 tablet.cc:1620] T 514c4b61a86745f793a2375d09184475 P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:22.125905 2976 tablet_bootstrap.cc:654] T 6042767df75143fd9f873d51fefbc9f7 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:22.127020 2976 tablet_bootstrap.cc:492] T 6042767df75143fd9f873d51fefbc9f7 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:22.127214 2976 ts_tablet_manager.cc:1367] T 6042767df75143fd9f873d51fefbc9f7 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 20:46:22.127871 2976 raft_consensus.cc:348] T 6042767df75143fd9f873d51fefbc9f7 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:22.128038 2976 raft_consensus.cc:374] T 6042767df75143fd9f873d51fefbc9f7 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:22.128113 2976 raft_consensus.cc:729] T 6042767df75143fd9f873d51fefbc9f7 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:22.128409 2976 consensus_queue.cc:260] T 6042767df75143fd9f873d51fefbc9f7 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:22.128773 2976 ts_tablet_manager.cc:1398] T 6042767df75143fd9f873d51fefbc9f7 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:22.140576 3167 raft_consensus.cc:482] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:22.140705 3167 raft_consensus.cc:504] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:22.141139 3167 leader_election.cc:288] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:22.141507 31887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2227f1ae58a044ea955cffa8d8d8a3f2" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:22.141590 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2227f1ae58a044ea955cffa8d8d8a3f2" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:22.141657 31887 raft_consensus.cc:2454] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:22.141712 31756 raft_consensus.cc:2454] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:22.142014 31429 leader_election.cc:302] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:22.142181 3167 raft_consensus.cc:2781] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:22.142232 3167 raft_consensus.cc:482] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:22.142267 3167 raft_consensus.cc:3037] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:22.143178 3167 raft_consensus.cc:504] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:22.143503 3167 leader_election.cc:288] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Requested vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:22.143819 31887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2227f1ae58a044ea955cffa8d8d8a3f2" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:22.143918 31887 raft_consensus.cc:3037] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:22.144608 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2227f1ae58a044ea955cffa8d8d8a3f2" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:22.144716 31757 raft_consensus.cc:3037] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:22.144941 31887 raft_consensus.cc:2454] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:22.145429 31429 leader_election.cc:302] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:22.145579 3167 raft_consensus.cc:2781] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:22.145568 31757 raft_consensus.cc:2454] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:22.145713 3167 raft_consensus.cc:686] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Becoming Leader. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Running, Role: LEADER
I20240502 20:46:22.145845 3167 consensus_queue.cc:237] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:22.147734 29033 catalog_manager.cc:5484] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69). New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:22.153513 29314 consensus_queue.cc:237] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:22.154790 31495 raft_consensus.cc:1261] T 741da32f7c924490a5fa5933708adf30 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:22.154824 29444 raft_consensus.cc:1261] T 741da32f7c924490a5fa5933708adf30 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:22.155162 3325 consensus_queue.cc:1035] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:22.155357 3089 consensus_queue.cc:1035] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:22.156344 3289 raft_consensus.cc:482] T d6b1c1798aaf4f1fb3d6ef52503e08be P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:22.156435 3289 raft_consensus.cc:504] T d6b1c1798aaf4f1fb3d6ef52503e08be P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:22.156824 3289 leader_election.cc:288] T d6b1c1798aaf4f1fb3d6ef52503e08be P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:22.157181 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6b1c1798aaf4f1fb3d6ef52503e08be" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:22.157100 3089 raft_consensus.cc:2932] T 741da32f7c924490a5fa5933708adf30 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:22.157214 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6b1c1798aaf4f1fb3d6ef52503e08be" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:22.157277 31495 raft_consensus.cc:2454] T d6b1c1798aaf4f1fb3d6ef52503e08be P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:22.157351 31625 raft_consensus.cc:2454] T d6b1c1798aaf4f1fb3d6ef52503e08be P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:22.157544 31692 leader_election.cc:302] T d6b1c1798aaf4f1fb3d6ef52503e08be P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:22.157445 29444 raft_consensus.cc:2932] T 741da32f7c924490a5fa5933708adf30 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:22.157752 3289 raft_consensus.cc:2781] T d6b1c1798aaf4f1fb3d6ef52503e08be P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:22.157792 3289 raft_consensus.cc:482] T d6b1c1798aaf4f1fb3d6ef52503e08be P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:22.157821 3289 raft_consensus.cc:3037] T d6b1c1798aaf4f1fb3d6ef52503e08be P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:22.158891 3289 raft_consensus.cc:504] T d6b1c1798aaf4f1fb3d6ef52503e08be P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:22.159251 3289 leader_election.cc:288] T d6b1c1798aaf4f1fb3d6ef52503e08be P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Requested vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:22.159685 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6b1c1798aaf4f1fb3d6ef52503e08be" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:22.159762 31625 raft_consensus.cc:3037] T d6b1c1798aaf4f1fb3d6ef52503e08be P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:22.160332 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6b1c1798aaf4f1fb3d6ef52503e08be" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:22.160434 31494 raft_consensus.cc:3037] T d6b1c1798aaf4f1fb3d6ef52503e08be P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:22.160682 31625 raft_consensus.cc:2454] T d6b1c1798aaf4f1fb3d6ef52503e08be P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:22.160684 29036 catalog_manager.cc:5484] T 741da32f7c924490a5fa5933708adf30 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index 4 to 5, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) evicted. New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:22.161314 31494 raft_consensus.cc:2454] T d6b1c1798aaf4f1fb3d6ef52503e08be P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:22.161350 31691 leader_election.cc:302] T d6b1c1798aaf4f1fb3d6ef52503e08be P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:22.161489 3289 raft_consensus.cc:2781] T d6b1c1798aaf4f1fb3d6ef52503e08be P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:22.161576 3289 raft_consensus.cc:686] T d6b1c1798aaf4f1fb3d6ef52503e08be P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Becoming Leader. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Running, Role: LEADER
I20240502 20:46:22.161691 3289 consensus_queue.cc:237] T d6b1c1798aaf4f1fb3d6ef52503e08be P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:22.161674 31495 raft_consensus.cc:2932] T 741da32f7c924490a5fa5933708adf30 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:22.164330 29557 tablet_service.cc:1514] Processing DeleteTablet for tablet 741da32f7c924490a5fa5933708adf30 with delete_type TABLET_DATA_TOMBSTONED (TS db17f26a73f3447b92aa9a947e3a7057 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:22.165583 29034 catalog_manager.cc:5484] T d6b1c1798aaf4f1fb3d6ef52503e08be P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: term changed from 0 to 1, leader changed from <none> to e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71). New cstate: current_term: 1 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:22.165887 3330 tablet_replica.cc:330] T 741da32f7c924490a5fa5933708adf30 P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:22.166031 3330 raft_consensus.cc:2229] T 741da32f7c924490a5fa5933708adf30 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:22.166216 3330 raft_consensus.cc:2258] T 741da32f7c924490a5fa5933708adf30 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:22.166780 3330 ts_tablet_manager.cc:1875] T 741da32f7c924490a5fa5933708adf30 P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:22.169102 3330 ts_tablet_manager.cc:1888] T 741da32f7c924490a5fa5933708adf30 P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 20:46:22.169166 3330 log.cc:1196] T 741da32f7c924490a5fa5933708adf30 P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/741da32f7c924490a5fa5933708adf30
I20240502 20:46:22.169584 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet 741da32f7c924490a5fa5933708adf30 (table rebalance_test_table_9 [id=4b510e9cf32a4794a9193f7ebac46f93]) successfully deleted
I20240502 20:46:22.229869 29040 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:22.236652 3303 tablet.cc:1722] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:22.236964 3303 tablet.cc:1620] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:22.237326 3305 tablet.cc:1722] T 514c4b61a86745f793a2375d09184475 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:22.237610 3305 tablet.cc:1620] T 514c4b61a86745f793a2375d09184475 P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:22.239071 3304 tablet.cc:1722] T 514c4b61a86745f793a2375d09184475 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:22.239372 3304 tablet.cc:1620] T 514c4b61a86745f793a2375d09184475 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:22.245546 29034 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=be2a9e7ea3de486180ad95784329fbdb] alter complete (version 2)
I20240502 20:46:22.290073 3167 raft_consensus.cc:482] T 6042767df75143fd9f873d51fefbc9f7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:22.290205 3167 raft_consensus.cc:504] T 6042767df75143fd9f873d51fefbc9f7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:22.290606 3167 leader_election.cc:288] T 6042767df75143fd9f873d51fefbc9f7 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:22.290963 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6042767df75143fd9f873d51fefbc9f7" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:22.290964 31887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6042767df75143fd9f873d51fefbc9f7" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:22.291110 31757 raft_consensus.cc:2454] T 6042767df75143fd9f873d51fefbc9f7 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:22.291110 31887 raft_consensus.cc:2454] T 6042767df75143fd9f873d51fefbc9f7 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:22.291368 31429 leader_election.cc:302] T 6042767df75143fd9f873d51fefbc9f7 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:22.291534 3167 raft_consensus.cc:2781] T 6042767df75143fd9f873d51fefbc9f7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:22.291596 3167 raft_consensus.cc:482] T 6042767df75143fd9f873d51fefbc9f7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:22.291648 3167 raft_consensus.cc:3037] T 6042767df75143fd9f873d51fefbc9f7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:22.292474 3167 raft_consensus.cc:504] T 6042767df75143fd9f873d51fefbc9f7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:22.292771 3167 leader_election.cc:288] T 6042767df75143fd9f873d51fefbc9f7 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Requested vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:22.293076 31887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6042767df75143fd9f873d51fefbc9f7" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:22.293186 31887 raft_consensus.cc:3037] T 6042767df75143fd9f873d51fefbc9f7 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:22.294131 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6042767df75143fd9f873d51fefbc9f7" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:22.294160 31887 raft_consensus.cc:2454] T 6042767df75143fd9f873d51fefbc9f7 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:22.294250 31757 raft_consensus.cc:3037] T 6042767df75143fd9f873d51fefbc9f7 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:22.294517 31429 leader_election.cc:302] T 6042767df75143fd9f873d51fefbc9f7 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:22.294687 3167 raft_consensus.cc:2781] T 6042767df75143fd9f873d51fefbc9f7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:22.294749 3167 raft_consensus.cc:686] T 6042767df75143fd9f873d51fefbc9f7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Becoming Leader. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Running, Role: LEADER
I20240502 20:46:22.294852 3167 consensus_queue.cc:237] T 6042767df75143fd9f873d51fefbc9f7 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:22.295217 31757 raft_consensus.cc:2454] T 6042767df75143fd9f873d51fefbc9f7 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:22.296610 29034 catalog_manager.cc:5484] T 6042767df75143fd9f873d51fefbc9f7 P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69). New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:22.324214 29163 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:22.327078 29424 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:22.332507 29293 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:22.332954 29557 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:22.333278 31867 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:22.335580 31474 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:22.335842 31737 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:22.346545 29034 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:22.353663 31605 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:22.359112 3303 tablet.cc:1722] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:22.362105 3304 tablet.cc:1722] T 514c4b61a86745f793a2375d09184475 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:22.363898 3305 tablet.cc:1722] T 514c4b61a86745f793a2375d09184475 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:22.367388 29040 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=be2a9e7ea3de486180ad95784329fbdb] alter complete (version 3)
I20240502 20:46:22.429052 31887 consensus_queue.cc:237] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:22.430250 29315 raft_consensus.cc:1261] T 0f57649f122e45f9b8ef82ec975e5359 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:22.430374 31495 raft_consensus.cc:1261] T 0f57649f122e45f9b8ef82ec975e5359 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:22.430845 3284 consensus_queue.cc:1035] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:22.431054 2882 consensus_queue.cc:1035] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:22.432857 3284 raft_consensus.cc:2932] T 0f57649f122e45f9b8ef82ec975e5359 P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:22.433282 29315 raft_consensus.cc:2932] T 0f57649f122e45f9b8ef82ec975e5359 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:22.434311 31495 raft_consensus.cc:2932] T 0f57649f122e45f9b8ef82ec975e5359 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:22.435921 29034 catalog_manager.cc:5484] T 0f57649f122e45f9b8ef82ec975e5359 P 09258ac095664fa99caf0707262cde13 reported cstate change: config changed from index 2 to 3, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) evicted. New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:22.439927 31605 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f57649f122e45f9b8ef82ec975e5359 with delete_type TABLET_DATA_TOMBSTONED (TS 315915efcf2b413ca7f9d874338e1b2d not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:58346
I20240502 20:46:22.440281 3408 tablet_replica.cc:330] T 0f57649f122e45f9b8ef82ec975e5359 P 315915efcf2b413ca7f9d874338e1b2d: stopping tablet replica
I20240502 20:46:22.440436 3408 raft_consensus.cc:2229] T 0f57649f122e45f9b8ef82ec975e5359 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:22.440585 3408 raft_consensus.cc:2258] T 0f57649f122e45f9b8ef82ec975e5359 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:22.440987 3408 ts_tablet_manager.cc:1875] T 0f57649f122e45f9b8ef82ec975e5359 P 315915efcf2b413ca7f9d874338e1b2d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:22.442975 3408 ts_tablet_manager.cc:1888] T 0f57649f122e45f9b8ef82ec975e5359 P 315915efcf2b413ca7f9d874338e1b2d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 20:46:22.443063 3408 log.cc:1196] T 0f57649f122e45f9b8ef82ec975e5359 P 315915efcf2b413ca7f9d874338e1b2d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal/wals/0f57649f122e45f9b8ef82ec975e5359
I20240502 20:46:22.444454 29027 catalog_manager.cc:4830] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): tablet 0f57649f122e45f9b8ef82ec975e5359 (table rebalancer_extra_table_2 [id=536c055ebf024097b825854cd0490c67]) successfully deleted
I20240502 20:46:22.472594 29036 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:22.472738 29036 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:22.474967 29036 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 514c4b61a86745f793a2375d09184475
I20240502 20:46:22.475592 31737 tablet_service.cc:1514] Processing DeleteTablet for tablet 514c4b61a86745f793a2375d09184475 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:22 UTC) from {username='slave'} at 127.0.0.1:33864
I20240502 20:46:22.475602 31605 tablet_service.cc:1514] Processing DeleteTablet for tablet 514c4b61a86745f793a2375d09184475 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:22 UTC) from {username='slave'} at 127.0.0.1:58346
I20240502 20:46:22.475776 3408 tablet_replica.cc:330] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d: stopping tablet replica
I20240502 20:46:22.475893 3426 tablet_replica.cc:330] T 514c4b61a86745f793a2375d09184475 P e7ab6b9053664ee5aa71c7978e26fe6a: stopping tablet replica
I20240502 20:46:22.475914 3408 raft_consensus.cc:2229] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:22.476050 3426 raft_consensus.cc:2229] T 514c4b61a86745f793a2375d09184475 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:22.476137 3408 raft_consensus.cc:2258] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:22.476114 31867 tablet_service.cc:1514] Processing DeleteTablet for tablet 514c4b61a86745f793a2375d09184475 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:22 UTC) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:22.476161 3426 raft_consensus.cc:2258] T 514c4b61a86745f793a2375d09184475 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:22.476653 3426 ts_tablet_manager.cc:1875] T 514c4b61a86745f793a2375d09184475 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:22.476881 3408 ts_tablet_manager.cc:1875] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:22.479077 3426 ts_tablet_manager.cc:1888] T 514c4b61a86745f793a2375d09184475 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:22.479142 3408 ts_tablet_manager.cc:1888] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:22.479226 3408 log.cc:1196] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal/wals/514c4b61a86745f793a2375d09184475
I20240502 20:46:22.479208 3426 log.cc:1196] T 514c4b61a86745f793a2375d09184475 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/wals/514c4b61a86745f793a2375d09184475
I20240502 20:46:22.479496 3426 ts_tablet_manager.cc:1909] T 514c4b61a86745f793a2375d09184475 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting consensus metadata
I20240502 20:46:22.479496 3408 ts_tablet_manager.cc:1909] T 514c4b61a86745f793a2375d09184475 P 315915efcf2b413ca7f9d874338e1b2d: Deleting consensus metadata
I20240502 20:46:22.480036 29026 catalog_manager.cc:4830] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): tablet 514c4b61a86745f793a2375d09184475 (table rebalancer_dynamic_table_new_name [id=be2a9e7ea3de486180ad95784329fbdb]) successfully deleted
I20240502 20:46:22.480211 29027 catalog_manager.cc:4830] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): tablet 514c4b61a86745f793a2375d09184475 (table rebalancer_dynamic_table_new_name [id=be2a9e7ea3de486180ad95784329fbdb]) successfully deleted
I20240502 20:46:22.481992 3427 tablet_replica.cc:330] T 514c4b61a86745f793a2375d09184475 P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:22.482184 3427 raft_consensus.cc:2229] T 514c4b61a86745f793a2375d09184475 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:22.482787 3427 raft_consensus.cc:2258] T 514c4b61a86745f793a2375d09184475 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:22.483376 3427 ts_tablet_manager.cc:1875] T 514c4b61a86745f793a2375d09184475 P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:22.486721 3427 ts_tablet_manager.cc:1888] T 514c4b61a86745f793a2375d09184475 P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:22.486780 3427 log.cc:1196] T 514c4b61a86745f793a2375d09184475 P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/514c4b61a86745f793a2375d09184475
I20240502 20:46:22.487004 3427 ts_tablet_manager.cc:1909] T 514c4b61a86745f793a2375d09184475 P e6b60803d0764d228c863e2da65e3691: Deleting consensus metadata
I20240502 20:46:22.487563 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet 514c4b61a86745f793a2375d09184475 (table rebalancer_dynamic_table_new_name [id=be2a9e7ea3de486180ad95784329fbdb]) successfully deleted
I20240502 20:46:22.522711 29293 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:22.531090 29424 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:22.531087 31474 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:22.531960 31605 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:22.533154 29163 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:22.537729 29557 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:22.564396 31867 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:22.577488 31737 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:22.577445 29042 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:22.583796 31867 tablet_service.cc:1467] Processing CreateTablet for tablet ed4e0b529166405f871ab85abd63ba1e (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a6cebd03a491492cb465683bdb44f87d]), partition=
I20240502 20:46:22.584059 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed4e0b529166405f871ab85abd63ba1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:22.586025 31605 tablet_service.cc:1467] Processing CreateTablet for tablet ed4e0b529166405f871ab85abd63ba1e (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a6cebd03a491492cb465683bdb44f87d]), partition=
I20240502 20:46:22.586210 2736 tablet_bootstrap.cc:492] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:22.586269 31605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed4e0b529166405f871ab85abd63ba1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:22.587132 2736 tablet_bootstrap.cc:654] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:22.588342 3146 tablet_bootstrap.cc:492] T ed4e0b529166405f871ab85abd63ba1e P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:22.588601 2736 tablet_bootstrap.cc:492] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:22.588692 2736 ts_tablet_manager.cc:1367] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:22.589095 2736 raft_consensus.cc:348] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:22.589172 3146 tablet_bootstrap.cc:654] T ed4e0b529166405f871ab85abd63ba1e P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:22.589185 2736 raft_consensus.cc:374] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:22.589212 2736 raft_consensus.cc:729] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:22.589313 2736 consensus_queue.cc:260] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:22.589546 2736 ts_tablet_manager.cc:1398] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:22.589915 31737 tablet_service.cc:1467] Processing CreateTablet for tablet ed4e0b529166405f871ab85abd63ba1e (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a6cebd03a491492cb465683bdb44f87d]), partition=
I20240502 20:46:22.590111 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed4e0b529166405f871ab85abd63ba1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:22.592097 2976 tablet_bootstrap.cc:492] T ed4e0b529166405f871ab85abd63ba1e P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:22.592967 2976 tablet_bootstrap.cc:654] T ed4e0b529166405f871ab85abd63ba1e P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:22.594254 2976 tablet_bootstrap.cc:492] T ed4e0b529166405f871ab85abd63ba1e P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:22.594329 2976 ts_tablet_manager.cc:1367] T ed4e0b529166405f871ab85abd63ba1e P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 20:46:22.594889 2976 raft_consensus.cc:348] T ed4e0b529166405f871ab85abd63ba1e P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:22.594987 2976 raft_consensus.cc:374] T ed4e0b529166405f871ab85abd63ba1e P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:22.595012 2976 raft_consensus.cc:729] T ed4e0b529166405f871ab85abd63ba1e P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:22.595119 2976 consensus_queue.cc:260] T ed4e0b529166405f871ab85abd63ba1e P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:22.595382 2976 ts_tablet_manager.cc:1398] T ed4e0b529166405f871ab85abd63ba1e P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:22.600126 3146 tablet_bootstrap.cc:492] T ed4e0b529166405f871ab85abd63ba1e P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:22.600212 3146 ts_tablet_manager.cc:1367] T ed4e0b529166405f871ab85abd63ba1e P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.001s
I20240502 20:46:22.600605 3146 raft_consensus.cc:348] T ed4e0b529166405f871ab85abd63ba1e P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:22.600729 3146 raft_consensus.cc:374] T ed4e0b529166405f871ab85abd63ba1e P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:22.600754 3146 raft_consensus.cc:729] T ed4e0b529166405f871ab85abd63ba1e P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:22.600860 3146 consensus_queue.cc:260] T ed4e0b529166405f871ab85abd63ba1e P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:22.601127 3146 ts_tablet_manager.cc:1398] T ed4e0b529166405f871ab85abd63ba1e P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:22.611446 3167 consensus_queue.cc:1035] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:22.615180 3317 consensus_queue.cc:1035] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:22.631778 29444 consensus_queue.cc:237] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:22.632939 31888 raft_consensus.cc:1261] T a2bf196bcab440c48aa4ea893997573b P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:22.633077 29315 raft_consensus.cc:1261] T a2bf196bcab440c48aa4ea893997573b P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:22.633497 3480 consensus_queue.cc:1035] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:22.633702 3224 consensus_queue.cc:1035] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:22.635339 3224 raft_consensus.cc:2932] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:22.635742 31888 raft_consensus.cc:2932] T a2bf196bcab440c48aa4ea893997573b P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:22.636055 29315 raft_consensus.cc:2932] T a2bf196bcab440c48aa4ea893997573b P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:22.638264 29042 catalog_manager.cc:5484] T a2bf196bcab440c48aa4ea893997573b P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index 4 to 5, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) evicted. New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:22.640856 29557 tablet_service.cc:1514] Processing DeleteTablet for tablet a2bf196bcab440c48aa4ea893997573b with delete_type TABLET_DATA_TOMBSTONED (TS db17f26a73f3447b92aa9a947e3a7057 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:22.641110 3330 tablet_replica.cc:330] T a2bf196bcab440c48aa4ea893997573b P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:22.641261 3330 raft_consensus.cc:2229] T a2bf196bcab440c48aa4ea893997573b P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:22.641414 3330 raft_consensus.cc:2258] T a2bf196bcab440c48aa4ea893997573b P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:22.641989 3330 ts_tablet_manager.cc:1875] T a2bf196bcab440c48aa4ea893997573b P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:22.643182 3289 consensus_queue.cc:1035] T d6b1c1798aaf4f1fb3d6ef52503e08be P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:22.644368 3330 ts_tablet_manager.cc:1888] T a2bf196bcab440c48aa4ea893997573b P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 20:46:22.644448 3330 log.cc:1196] T a2bf196bcab440c48aa4ea893997573b P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/a2bf196bcab440c48aa4ea893997573b
I20240502 20:46:22.644994 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet a2bf196bcab440c48aa4ea893997573b (table rebalance_test_table_3 [id=aec26ee01a7a4b05887071d6b1783fb0]) successfully deleted
I20240502 20:46:22.646483 3289 consensus_queue.cc:1035] T d6b1c1798aaf4f1fb3d6ef52503e08be P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:22.813997 3284 raft_consensus.cc:482] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:22.814122 3284 raft_consensus.cc:504] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:22.814546 3284 leader_election.cc:288] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:22.814819 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed4e0b529166405f871ab85abd63ba1e" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:22.814930 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed4e0b529166405f871ab85abd63ba1e" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:22.814970 31625 raft_consensus.cc:2454] T ed4e0b529166405f871ab85abd63ba1e P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:22.815068 31756 raft_consensus.cc:2454] T ed4e0b529166405f871ab85abd63ba1e P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:22.815275 31822 leader_election.cc:302] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:22.815454 3284 raft_consensus.cc:2781] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:22.815521 3284 raft_consensus.cc:482] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:22.815548 3284 raft_consensus.cc:3037] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:22.820071 3284 raft_consensus.cc:504] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:22.820436 3284 leader_election.cc:288] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:22.820742 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed4e0b529166405f871ab85abd63ba1e" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:22.820827 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed4e0b529166405f871ab85abd63ba1e" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:22.820869 31625 raft_consensus.cc:3037] T ed4e0b529166405f871ab85abd63ba1e P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:22.820942 31756 raft_consensus.cc:3037] T ed4e0b529166405f871ab85abd63ba1e P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:22.821878 31625 raft_consensus.cc:2454] T ed4e0b529166405f871ab85abd63ba1e P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:22.821877 31756 raft_consensus.cc:2454] T ed4e0b529166405f871ab85abd63ba1e P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:22.822144 31822 leader_election.cc:302] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:22.822373 3284 raft_consensus.cc:2781] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:22.822485 3284 raft_consensus.cc:686] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Becoming Leader. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Running, Role: LEADER
I20240502 20:46:22.822621 3284 consensus_queue.cc:237] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:22.824620 29042 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_28"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:22.824641 29036 catalog_manager.cc:5484] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6b60803d0764d228c863e2da65e3691 (127.22.45.72). New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:22.830149 31737 tablet_service.cc:1467] Processing CreateTablet for tablet d5ca50fc288f4957a2d6f04c23bd1491 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=5dbd6a79afdc419e96698104b5e310ab]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:22.830323 29293 tablet_service.cc:1467] Processing CreateTablet for tablet d5ca50fc288f4957a2d6f04c23bd1491 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=5dbd6a79afdc419e96698104b5e310ab]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:22.830426 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5ca50fc288f4957a2d6f04c23bd1491. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:22.830631 31867 tablet_service.cc:1467] Processing CreateTablet for tablet d5ca50fc288f4957a2d6f04c23bd1491 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=5dbd6a79afdc419e96698104b5e310ab]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:22.830698 29293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5ca50fc288f4957a2d6f04c23bd1491. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:22.830901 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5ca50fc288f4957a2d6f04c23bd1491. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:22.832676 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 4aa8ea08d2b247b884cc14437d1441a8 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=5dbd6a79afdc419e96698104b5e310ab]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:22.832841 29163 tablet_service.cc:1467] Processing CreateTablet for tablet 3d89ea92aec54168997241dd78c6abb1 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=5dbd6a79afdc419e96698104b5e310ab]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:22.832926 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aa8ea08d2b247b884cc14437d1441a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:22.832981 31735 tablet_service.cc:1467] Processing CreateTablet for tablet 4aa8ea08d2b247b884cc14437d1441a8 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=5dbd6a79afdc419e96698104b5e310ab]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:22.833119 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d89ea92aec54168997241dd78c6abb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:22.833179 31735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aa8ea08d2b247b884cc14437d1441a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:22.834585 2736 tablet_bootstrap.cc:492] T d5ca50fc288f4957a2d6f04c23bd1491 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:22.834610 2976 tablet_bootstrap.cc:492] T d5ca50fc288f4957a2d6f04c23bd1491 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:22.835525 2976 tablet_bootstrap.cc:654] T d5ca50fc288f4957a2d6f04c23bd1491 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:22.835655 2736 tablet_bootstrap.cc:654] T d5ca50fc288f4957a2d6f04c23bd1491 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:22.837049 31605 tablet_service.cc:1467] Processing CreateTablet for tablet 3d89ea92aec54168997241dd78c6abb1 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=5dbd6a79afdc419e96698104b5e310ab]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:22.837301 31605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d89ea92aec54168997241dd78c6abb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:22.839395 3146 tablet_bootstrap.cc:492] T 3d89ea92aec54168997241dd78c6abb1 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:22.839689 29424 tablet_service.cc:1467] Processing CreateTablet for tablet 3d89ea92aec54168997241dd78c6abb1 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=5dbd6a79afdc419e96698104b5e310ab]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:22.839867 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 4aa8ea08d2b247b884cc14437d1441a8 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=5dbd6a79afdc419e96698104b5e310ab]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:22.839953 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d89ea92aec54168997241dd78c6abb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:22.840090 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aa8ea08d2b247b884cc14437d1441a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:22.840281 3146 tablet_bootstrap.cc:654] T 3d89ea92aec54168997241dd78c6abb1 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:22.841599 3146 tablet_bootstrap.cc:492] T 3d89ea92aec54168997241dd78c6abb1 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:22.841686 3146 ts_tablet_manager.cc:1367] T 3d89ea92aec54168997241dd78c6abb1 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 20:46:22.842115 3146 raft_consensus.cc:348] T 3d89ea92aec54168997241dd78c6abb1 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:22.842226 3146 raft_consensus.cc:374] T 3d89ea92aec54168997241dd78c6abb1 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:22.842257 3146 raft_consensus.cc:729] T 3d89ea92aec54168997241dd78c6abb1 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:22.842365 3146 consensus_queue.cc:260] T 3d89ea92aec54168997241dd78c6abb1 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:22.842643 3146 ts_tablet_manager.cc:1398] T 3d89ea92aec54168997241dd78c6abb1 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:22.842670 2976 tablet_bootstrap.cc:492] T d5ca50fc288f4957a2d6f04c23bd1491 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:22.842733 2976 ts_tablet_manager.cc:1367] T d5ca50fc288f4957a2d6f04c23bd1491 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 20:46:22.843147 2976 raft_consensus.cc:348] T d5ca50fc288f4957a2d6f04c23bd1491 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:22.843243 2976 raft_consensus.cc:374] T d5ca50fc288f4957a2d6f04c23bd1491 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:22.843281 2976 raft_consensus.cc:729] T d5ca50fc288f4957a2d6f04c23bd1491 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:22.843401 2976 consensus_queue.cc:260] T d5ca50fc288f4957a2d6f04c23bd1491 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:22.843695 2976 ts_tablet_manager.cc:1398] T d5ca50fc288f4957a2d6f04c23bd1491 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:22.843878 2976 tablet_bootstrap.cc:492] T 4aa8ea08d2b247b884cc14437d1441a8 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:22.845229 2976 tablet_bootstrap.cc:654] T 4aa8ea08d2b247b884cc14437d1441a8 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:22.845356 3546 tablet_bootstrap.cc:492] T 3d89ea92aec54168997241dd78c6abb1 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:22.846266 3546 tablet_bootstrap.cc:654] T 3d89ea92aec54168997241dd78c6abb1 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:22.847261 3549 tablet_bootstrap.cc:492] T 3d89ea92aec54168997241dd78c6abb1 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:22.847410 2736 tablet_bootstrap.cc:492] T d5ca50fc288f4957a2d6f04c23bd1491 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:22.847501 2736 ts_tablet_manager.cc:1367] T d5ca50fc288f4957a2d6f04c23bd1491 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 20:46:22.847889 2736 raft_consensus.cc:348] T d5ca50fc288f4957a2d6f04c23bd1491 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:22.847975 2736 raft_consensus.cc:374] T d5ca50fc288f4957a2d6f04c23bd1491 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:22.848003 2736 raft_consensus.cc:729] T d5ca50fc288f4957a2d6f04c23bd1491 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:22.848106 2736 consensus_queue.cc:260] T d5ca50fc288f4957a2d6f04c23bd1491 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:22.848244 3549 tablet_bootstrap.cc:654] T 3d89ea92aec54168997241dd78c6abb1 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:22.848361 2736 ts_tablet_manager.cc:1398] T d5ca50fc288f4957a2d6f04c23bd1491 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:22.848515 2736 tablet_bootstrap.cc:492] T 4aa8ea08d2b247b884cc14437d1441a8 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:22.849252 3546 tablet_bootstrap.cc:492] T 3d89ea92aec54168997241dd78c6abb1 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:22.849332 3546 ts_tablet_manager.cc:1367] T 3d89ea92aec54168997241dd78c6abb1 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:22.849345 2736 tablet_bootstrap.cc:654] T 4aa8ea08d2b247b884cc14437d1441a8 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:22.849727 3546 raft_consensus.cc:348] T 3d89ea92aec54168997241dd78c6abb1 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:22.849838 3546 raft_consensus.cc:374] T 3d89ea92aec54168997241dd78c6abb1 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:22.849877 3546 raft_consensus.cc:729] T 3d89ea92aec54168997241dd78c6abb1 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:22.850190 3534 tablet_bootstrap.cc:492] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:22.850445 2736 tablet_bootstrap.cc:492] T 4aa8ea08d2b247b884cc14437d1441a8 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:22.850535 2736 ts_tablet_manager.cc:1367] T 4aa8ea08d2b247b884cc14437d1441a8 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:22.850486 3546 consensus_queue.cc:260] T 3d89ea92aec54168997241dd78c6abb1 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:22.850755 3546 ts_tablet_manager.cc:1398] T 3d89ea92aec54168997241dd78c6abb1 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:22.850957 2736 raft_consensus.cc:348] T 4aa8ea08d2b247b884cc14437d1441a8 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:22.851042 2736 raft_consensus.cc:374] T 4aa8ea08d2b247b884cc14437d1441a8 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:22.851068 2736 raft_consensus.cc:729] T 4aa8ea08d2b247b884cc14437d1441a8 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:22.851166 2736 consensus_queue.cc:260] T 4aa8ea08d2b247b884cc14437d1441a8 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:22.851393 2736 ts_tablet_manager.cc:1398] T 4aa8ea08d2b247b884cc14437d1441a8 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:22.852223 3534 tablet_bootstrap.cc:654] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:22.852566 3549 tablet_bootstrap.cc:492] T 3d89ea92aec54168997241dd78c6abb1 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:22.852711 3549 ts_tablet_manager.cc:1367] T 3d89ea92aec54168997241dd78c6abb1 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 20:46:22.853199 3549 raft_consensus.cc:348] T 3d89ea92aec54168997241dd78c6abb1 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:22.854452 3549 raft_consensus.cc:374] T 3d89ea92aec54168997241dd78c6abb1 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:22.854563 3549 raft_consensus.cc:729] T 3d89ea92aec54168997241dd78c6abb1 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:22.855227 29295 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 20:46:22.855995 2976 tablet_bootstrap.cc:492] T 4aa8ea08d2b247b884cc14437d1441a8 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:22.856097 2976 ts_tablet_manager.cc:1367] T 4aa8ea08d2b247b884cc14437d1441a8 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 20:46:22.856127 3534 tablet_bootstrap.cc:492] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:22.856220 3534 ts_tablet_manager.cc:1367] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 20:46:22.856385 3549 consensus_queue.cc:260] T 3d89ea92aec54168997241dd78c6abb1 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:22.856559 2976 raft_consensus.cc:348] T 4aa8ea08d2b247b884cc14437d1441a8 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:22.856662 2976 raft_consensus.cc:374] T 4aa8ea08d2b247b884cc14437d1441a8 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:22.856703 2976 raft_consensus.cc:729] T 4aa8ea08d2b247b884cc14437d1441a8 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:22.856632 3534 raft_consensus.cc:348] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:22.856823 2976 consensus_queue.cc:260] T 4aa8ea08d2b247b884cc14437d1441a8 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:22.856873 3534 raft_consensus.cc:374] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:22.857013 3549 ts_tablet_manager.cc:1398] T 3d89ea92aec54168997241dd78c6abb1 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:22.857090 2976 ts_tablet_manager.cc:1398] T 4aa8ea08d2b247b884cc14437d1441a8 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:22.856920 3534 raft_consensus.cc:729] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:22.858120 3534 consensus_queue.cc:260] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:22.858412 3534 ts_tablet_manager.cc:1398] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:22.859699 3534 tablet_bootstrap.cc:492] T 4aa8ea08d2b247b884cc14437d1441a8 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:22.860605 3534 tablet_bootstrap.cc:654] T 4aa8ea08d2b247b884cc14437d1441a8 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:22.862870 3534 tablet_bootstrap.cc:492] T 4aa8ea08d2b247b884cc14437d1441a8 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:22.863279 3534 ts_tablet_manager.cc:1367] T 4aa8ea08d2b247b884cc14437d1441a8 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 20:46:22.864830 3317 consensus_queue.cc:1035] T 6042767df75143fd9f873d51fefbc9f7 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:22.865494 3534 raft_consensus.cc:348] T 4aa8ea08d2b247b884cc14437d1441a8 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:22.866331 3534 raft_consensus.cc:374] T 4aa8ea08d2b247b884cc14437d1441a8 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:22.866427 3534 raft_consensus.cc:729] T 4aa8ea08d2b247b884cc14437d1441a8 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:22.867883 3534 consensus_queue.cc:260] T 4aa8ea08d2b247b884cc14437d1441a8 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:22.868471 3534 ts_tablet_manager.cc:1398] T 4aa8ea08d2b247b884cc14437d1441a8 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20240502 20:46:22.870553 31474 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:22.875030 3317 consensus_queue.cc:1035] T 6042767df75143fd9f873d51fefbc9f7 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:22.879022 31867 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:22.880853 29557 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:22.883625 29424 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:22.890998 29163 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:22.908152 31605 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:22.916963 31735 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:22.951679 3325 raft_consensus.cc:482] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:22.951797 3325 raft_consensus.cc:504] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:22.952215 3325 leader_election.cc:288] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:22.952632 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5ca50fc288f4957a2d6f04c23bd1491" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:22.952646 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5ca50fc288f4957a2d6f04c23bd1491" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:22.952749 31888 raft_consensus.cc:2454] T d5ca50fc288f4957a2d6f04c23bd1491 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:22.952785 31756 raft_consensus.cc:2454] T d5ca50fc288f4957a2d6f04c23bd1491 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:22.953012 29249 leader_election.cc:302] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:22.953214 3325 raft_consensus.cc:2781] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:22.953259 3325 raft_consensus.cc:482] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:22.953294 3325 raft_consensus.cc:3037] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:22.954252 3325 raft_consensus.cc:504] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:22.954623 3325 leader_election.cc:288] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:22.955008 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5ca50fc288f4957a2d6f04c23bd1491" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:22.955106 31756 raft_consensus.cc:3037] T d5ca50fc288f4957a2d6f04c23bd1491 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:22.955896 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5ca50fc288f4957a2d6f04c23bd1491" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:22.955989 31888 raft_consensus.cc:3037] T d5ca50fc288f4957a2d6f04c23bd1491 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:22.956068 31756 raft_consensus.cc:2454] T d5ca50fc288f4957a2d6f04c23bd1491 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:22.956503 29248 leader_election.cc:302] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:22.956693 3325 raft_consensus.cc:2781] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:22.956709 31888 raft_consensus.cc:2454] T d5ca50fc288f4957a2d6f04c23bd1491 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:22.956784 3325 raft_consensus.cc:686] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:22.956908 3325 consensus_queue.cc:237] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:22.959254 29036 catalog_manager.cc:5484] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:22.969617 2144 raft_consensus.cc:482] T 3d89ea92aec54168997241dd78c6abb1 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:22.969743 2144 raft_consensus.cc:504] T 3d89ea92aec54168997241dd78c6abb1 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:22.970221 2144 leader_election.cc:288] T 3d89ea92aec54168997241dd78c6abb1 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:22.970618 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d89ea92aec54168997241dd78c6abb1" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:22.970772 29444 raft_consensus.cc:2454] T 3d89ea92aec54168997241dd78c6abb1 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:22.971086 29119 leader_election.cc:302] T 3d89ea92aec54168997241dd78c6abb1 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:22.971619 2144 raft_consensus.cc:2781] T 3d89ea92aec54168997241dd78c6abb1 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:22.971679 2144 raft_consensus.cc:482] T 3d89ea92aec54168997241dd78c6abb1 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:22.971704 2144 raft_consensus.cc:3037] T 3d89ea92aec54168997241dd78c6abb1 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:22.972656 2144 raft_consensus.cc:504] T 3d89ea92aec54168997241dd78c6abb1 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:22.972967 2144 leader_election.cc:288] T 3d89ea92aec54168997241dd78c6abb1 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:22.974076 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d89ea92aec54168997241dd78c6abb1" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:22.974196 29444 raft_consensus.cc:3037] T 3d89ea92aec54168997241dd78c6abb1 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:22.975137 29444 raft_consensus.cc:2454] T 3d89ea92aec54168997241dd78c6abb1 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:22.975394 29119 leader_election.cc:302] T 3d89ea92aec54168997241dd78c6abb1 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:22.975534 2144 raft_consensus.cc:2781] T 3d89ea92aec54168997241dd78c6abb1 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:22.975607 2144 raft_consensus.cc:686] T 3d89ea92aec54168997241dd78c6abb1 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:22.975732 2144 consensus_queue.cc:237] T 3d89ea92aec54168997241dd78c6abb1 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:22.976697 31888 consensus_queue.cc:237] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:22.976882 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d89ea92aec54168997241dd78c6abb1" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:22.977002 31625 raft_consensus.cc:2454] T 3d89ea92aec54168997241dd78c6abb1 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:22.977195 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d89ea92aec54168997241dd78c6abb1" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:22.977295 31626 raft_consensus.cc:3037] T 3d89ea92aec54168997241dd78c6abb1 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:22.978060 29577 raft_consensus.cc:1261] T 075d3240e24a4532bbb29a068c00c128 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:22.978163 31626 raft_consensus.cc:2454] T 3d89ea92aec54168997241dd78c6abb1 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:22.978718 29182 raft_consensus.cc:1261] T 075d3240e24a4532bbb29a068c00c128 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:22.979027 3400 consensus_queue.cc:1035] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:22.979410 29036 catalog_manager.cc:5484] T 3d89ea92aec54168997241dd78c6abb1 P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:22.980683 3400 consensus_queue.cc:1035] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:22.981796 3400 raft_consensus.cc:2932] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:22.982120 29577 raft_consensus.cc:2932] T 075d3240e24a4532bbb29a068c00c128 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:22.984836 29036 catalog_manager.cc:5484] T 075d3240e24a4532bbb29a068c00c128 P e6b60803d0764d228c863e2da65e3691 reported cstate change: config changed from index 2 to 3, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) evicted. New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:22.986013 29175 raft_consensus.cc:2932] T 075d3240e24a4532bbb29a068c00c128 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:22.986723 29042 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:22.988050 31735 tablet_service.cc:1514] Processing DeleteTablet for tablet 075d3240e24a4532bbb29a068c00c128 with delete_type TABLET_DATA_TOMBSTONED (TS e7ab6b9053664ee5aa71c7978e26fe6a not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:33864
I20240502 20:46:22.988373 3581 tablet_replica.cc:330] T 075d3240e24a4532bbb29a068c00c128 P e7ab6b9053664ee5aa71c7978e26fe6a: stopping tablet replica
I20240502 20:46:22.988531 3581 raft_consensus.cc:2229] T 075d3240e24a4532bbb29a068c00c128 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:22.988675 3581 raft_consensus.cc:2258] T 075d3240e24a4532bbb29a068c00c128 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:22.989121 3581 ts_tablet_manager.cc:1875] T 075d3240e24a4532bbb29a068c00c128 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:22.992523 31756 raft_consensus.cc:1261] T ed4e0b529166405f871ab85abd63ba1e P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:22.992581 31626 raft_consensus.cc:1261] T ed4e0b529166405f871ab85abd63ba1e P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:22.992924 3400 consensus_queue.cc:1035] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:22.993093 3284 consensus_queue.cc:1035] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:22.995275 29042 catalog_manager.cc:5576] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751) does not have the latest schema for tablet ed4e0b529166405f871ab85abd63ba1e (table rebalancer_dynamic_table [id=a6cebd03a491492cb465683bdb44f87d]). Expected version 1 got 0
I20240502 20:46:22.995971 29037 catalog_manager.cc:5576] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787) does not have the latest schema for tablet ed4e0b529166405f871ab85abd63ba1e (table rebalancer_dynamic_table [id=a6cebd03a491492cb465683bdb44f87d]). Expected version 1 got 0
I20240502 20:46:22.998129 3589 tablet.cc:1722] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:22.998342 3590 tablet.cc:1722] T ed4e0b529166405f871ab85abd63ba1e P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:22.998620 3589 tablet.cc:1620] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:22.998618 3590 tablet.cc:1620] T ed4e0b529166405f871ab85abd63ba1e P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:22.998692 29042 catalog_manager.cc:5576] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787) does not have the latest schema for tablet ed4e0b529166405f871ab85abd63ba1e (table rebalancer_dynamic_table [id=a6cebd03a491492cb465683bdb44f87d]). Expected version 1 got 0
I20240502 20:46:23.000308 29037 catalog_manager.cc:5576] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751) does not have the latest schema for tablet ed4e0b529166405f871ab85abd63ba1e (table rebalancer_dynamic_table [id=a6cebd03a491492cb465683bdb44f87d]). Expected version 1 got 0
I20240502 20:46:23.002946 29042 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=a6cebd03a491492cb465683bdb44f87d] alter complete (version 1)
I20240502 20:46:23.003657 3591 tablet.cc:1722] T ed4e0b529166405f871ab85abd63ba1e P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:23.004221 3591 tablet.cc:1620] T ed4e0b529166405f871ab85abd63ba1e P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:23.009186 3581 ts_tablet_manager.cc:1888] T 075d3240e24a4532bbb29a068c00c128 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 20:46:23.009271 3581 log.cc:1196] T 075d3240e24a4532bbb29a068c00c128 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/wals/075d3240e24a4532bbb29a068c00c128
I20240502 20:46:23.010355 29026 catalog_manager.cc:4830] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): tablet 075d3240e24a4532bbb29a068c00c128 (table rebalancer_extra_table_2 [id=536c055ebf024097b825854cd0490c67]) successfully deleted
I20240502 20:46:23.033712 31626 consensus_queue.cc:237] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:23.035037 31756 raft_consensus.cc:1261] T 90bab3d11d424b3abcdc74d5da51cbee P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer 315915efcf2b413ca7f9d874338e1b2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:23.035112 31888 raft_consensus.cc:1261] T 90bab3d11d424b3abcdc74d5da51cbee P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer 315915efcf2b413ca7f9d874338e1b2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 20:46:23.035248 31561 consensus_peers.cc:485] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d -> Peer dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): Couldn't send request to peer dbcdc6ac635b44b185c3eba63f66e9d6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 90bab3d11d424b3abcdc74d5da51cbee. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:23.035465 2916 consensus_queue.cc:1035] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:23.035656 3619 consensus_queue.cc:1035] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:23.037449 3619 raft_consensus.cc:2932] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:23.037876 31756 raft_consensus.cc:2932] T 90bab3d11d424b3abcdc74d5da51cbee P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:23.040663 29037 catalog_manager.cc:5484] T 90bab3d11d424b3abcdc74d5da51cbee P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: config changed from index -1 to 2, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) added. New cstate: current_term: 1 leader_uuid: "315915efcf2b413ca7f9d874338e1b2d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:23.040941 31888 raft_consensus.cc:2932] T 90bab3d11d424b3abcdc74d5da51cbee P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:23.058600 29175 consensus_queue.cc:237] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:23.060052 29444 raft_consensus.cc:1261] T 7fd046a0f0004e5fa6e0825cb626c2ff P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 20:46:23.060310 29117 consensus_peers.cc:485] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d -> Peer e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): Couldn't send request to peer e7ab6b9053664ee5aa71c7978e26fe6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7fd046a0f0004e5fa6e0825cb626c2ff. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:23.060479 2144 consensus_queue.cc:1035] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:23.060530 29315 raft_consensus.cc:1261] T 7fd046a0f0004e5fa6e0825cb626c2ff P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:23.060981 2144 consensus_queue.cc:1035] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:23.062848 3639 raft_consensus.cc:2932] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:23.063266 29315 raft_consensus.cc:2932] T 7fd046a0f0004e5fa6e0825cb626c2ff P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:23.064842 29444 raft_consensus.cc:2932] T 7fd046a0f0004e5fa6e0825cb626c2ff P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:23.066017 29037 catalog_manager.cc:5484] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d reported cstate change: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:23.107930 29042 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:23.116508 3589 tablet.cc:1722] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:23.116847 3589 tablet.cc:1620] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:23.117393 3590 tablet.cc:1722] T ed4e0b529166405f871ab85abd63ba1e P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:23.117671 3590 tablet.cc:1620] T ed4e0b529166405f871ab85abd63ba1e P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:23.120882 3591 tablet.cc:1722] T ed4e0b529166405f871ab85abd63ba1e P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:23.121196 3591 tablet.cc:1620] T ed4e0b529166405f871ab85abd63ba1e P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:23.122592 29034 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=a6cebd03a491492cb465683bdb44f87d] alter complete (version 2)
I20240502 20:46:23.173164 3284 raft_consensus.cc:482] T 4aa8ea08d2b247b884cc14437d1441a8 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:23.173272 3284 raft_consensus.cc:504] T 4aa8ea08d2b247b884cc14437d1441a8 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:23.173668 3284 leader_election.cc:288] T 4aa8ea08d2b247b884cc14437d1441a8 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:23.174022 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aa8ea08d2b247b884cc14437d1441a8" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:23.174131 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aa8ea08d2b247b884cc14437d1441a8" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:23.174188 31756 raft_consensus.cc:2454] T 4aa8ea08d2b247b884cc14437d1441a8 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:23.174288 29315 raft_consensus.cc:2454] T 4aa8ea08d2b247b884cc14437d1441a8 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:23.174659 31821 leader_election.cc:302] T 4aa8ea08d2b247b884cc14437d1441a8 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e6b60803d0764d228c863e2da65e3691, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:23.175089 3284 raft_consensus.cc:2781] T 4aa8ea08d2b247b884cc14437d1441a8 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:23.175149 3284 raft_consensus.cc:482] T 4aa8ea08d2b247b884cc14437d1441a8 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:23.175187 3284 raft_consensus.cc:3037] T 4aa8ea08d2b247b884cc14437d1441a8 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:23.176009 3284 raft_consensus.cc:504] T 4aa8ea08d2b247b884cc14437d1441a8 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:23.176313 3284 leader_election.cc:288] T 4aa8ea08d2b247b884cc14437d1441a8 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Requested vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:23.176599 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aa8ea08d2b247b884cc14437d1441a8" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:23.176645 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aa8ea08d2b247b884cc14437d1441a8" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:23.176705 31756 raft_consensus.cc:3037] T 4aa8ea08d2b247b884cc14437d1441a8 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:23.176755 29315 raft_consensus.cc:3037] T 4aa8ea08d2b247b884cc14437d1441a8 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:23.177466 29315 raft_consensus.cc:2454] T 4aa8ea08d2b247b884cc14437d1441a8 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:23.177630 31756 raft_consensus.cc:2454] T 4aa8ea08d2b247b884cc14437d1441a8 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:23.177716 31823 leader_election.cc:302] T 4aa8ea08d2b247b884cc14437d1441a8 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:23.177865 3284 raft_consensus.cc:2781] T 4aa8ea08d2b247b884cc14437d1441a8 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:23.177933 3284 raft_consensus.cc:686] T 4aa8ea08d2b247b884cc14437d1441a8 P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Becoming Leader. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Running, Role: LEADER
I20240502 20:46:23.178050 3284 consensus_queue.cc:237] T 4aa8ea08d2b247b884cc14437d1441a8 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:23.180009 29037 catalog_manager.cc:5484] T 4aa8ea08d2b247b884cc14437d1441a8 P e6b60803d0764d228c863e2da65e3691 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6b60803d0764d228c863e2da65e3691 (127.22.45.72). New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:23.204404 29163 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:23.205142 31474 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:23.206669 29295 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:23.208485 31605 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:23.211565 29424 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:23.214597 31867 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:23.216816 31735 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:23.222826 29557 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:23.223814 29037 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:23.237524 3589 tablet.cc:1722] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:23.238351 3590 tablet.cc:1722] T ed4e0b529166405f871ab85abd63ba1e P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:23.240495 3591 tablet.cc:1722] T ed4e0b529166405f871ab85abd63ba1e P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:23.246289 29035 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=a6cebd03a491492cb465683bdb44f87d] alter complete (version 3)
I20240502 20:46:23.344861 3531 consensus_queue.cc:1035] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:23.348477 3531 consensus_queue.cc:1035] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:23.350837 29037 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:23.350966 29037 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:23.355057 29037 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet ed4e0b529166405f871ab85abd63ba1e
I20240502 20:46:23.355738 31867 tablet_service.cc:1514] Processing DeleteTablet for tablet ed4e0b529166405f871ab85abd63ba1e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:23 UTC) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:23.355751 31605 tablet_service.cc:1514] Processing DeleteTablet for tablet ed4e0b529166405f871ab85abd63ba1e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:23 UTC) from {username='slave'} at 127.0.0.1:58346
I20240502 20:46:23.355983 31735 tablet_service.cc:1514] Processing DeleteTablet for tablet ed4e0b529166405f871ab85abd63ba1e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:23 UTC) from {username='slave'} at 127.0.0.1:33864
I20240502 20:46:23.356160 3581 tablet_replica.cc:330] T ed4e0b529166405f871ab85abd63ba1e P e7ab6b9053664ee5aa71c7978e26fe6a: stopping tablet replica
I20240502 20:46:23.356297 3581 raft_consensus.cc:2229] T ed4e0b529166405f871ab85abd63ba1e P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:23.356410 3581 raft_consensus.cc:2258] T ed4e0b529166405f871ab85abd63ba1e P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:23.357323 3581 ts_tablet_manager.cc:1875] T ed4e0b529166405f871ab85abd63ba1e P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:23.361280 3581 ts_tablet_manager.cc:1888] T ed4e0b529166405f871ab85abd63ba1e P e7ab6b9053664ee5aa71c7978e26fe6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:23.361389 3581 log.cc:1196] T ed4e0b529166405f871ab85abd63ba1e P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/wals/ed4e0b529166405f871ab85abd63ba1e
I20240502 20:46:23.361711 3581 ts_tablet_manager.cc:1909] T ed4e0b529166405f871ab85abd63ba1e P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting consensus metadata
I20240502 20:46:23.362273 3756 tablet_replica.cc:330] T ed4e0b529166405f871ab85abd63ba1e P 315915efcf2b413ca7f9d874338e1b2d: stopping tablet replica
I20240502 20:46:23.362351 29026 catalog_manager.cc:4830] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): tablet ed4e0b529166405f871ab85abd63ba1e (table rebalancer_dynamic_table_new_name [id=a6cebd03a491492cb465683bdb44f87d]) successfully deleted
I20240502 20:46:23.362622 3756 raft_consensus.cc:2229] T ed4e0b529166405f871ab85abd63ba1e P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:23.362910 3756 raft_consensus.cc:2258] T ed4e0b529166405f871ab85abd63ba1e P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:23.363140 3757 tablet_replica.cc:330] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:23.363283 3757 raft_consensus.cc:2229] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:23.363662 3757 raft_consensus.cc:2258] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:23.364339 3757 ts_tablet_manager.cc:1875] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:23.366380 31605 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:23.367305 3756 ts_tablet_manager.cc:1875] T ed4e0b529166405f871ab85abd63ba1e P 315915efcf2b413ca7f9d874338e1b2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:23.368322 29163 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:23.375891 31867 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:23.379009 3756 ts_tablet_manager.cc:1888] T ed4e0b529166405f871ab85abd63ba1e P 315915efcf2b413ca7f9d874338e1b2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:23.379151 3756 log.cc:1196] T ed4e0b529166405f871ab85abd63ba1e P 315915efcf2b413ca7f9d874338e1b2d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal/wals/ed4e0b529166405f871ab85abd63ba1e
I20240502 20:46:23.379530 3756 ts_tablet_manager.cc:1909] T ed4e0b529166405f871ab85abd63ba1e P 315915efcf2b413ca7f9d874338e1b2d: Deleting consensus metadata
I20240502 20:46:23.383150 29424 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:23.383775 29027 catalog_manager.cc:4830] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): tablet ed4e0b529166405f871ab85abd63ba1e (table rebalancer_dynamic_table_new_name [id=a6cebd03a491492cb465683bdb44f87d]) successfully deleted
I20240502 20:46:23.388716 29557 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:23.389948 1776 consensus_queue.cc:1035] T 3d89ea92aec54168997241dd78c6abb1 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:23.393239 31735 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:23.399255 31474 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:23.401412 29295 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:23.425071 3642 consensus_queue.cc:1035] T 3d89ea92aec54168997241dd78c6abb1 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:23.427708 3757 ts_tablet_manager.cc:1888] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:23.432356 3757 log.cc:1196] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/ed4e0b529166405f871ab85abd63ba1e
I20240502 20:46:23.432713 3757 ts_tablet_manager.cc:1909] T ed4e0b529166405f871ab85abd63ba1e P e6b60803d0764d228c863e2da65e3691: Deleting consensus metadata
I20240502 20:46:23.433741 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet ed4e0b529166405f871ab85abd63ba1e (table rebalancer_dynamic_table_new_name [id=a6cebd03a491492cb465683bdb44f87d]) successfully deleted
I20240502 20:46:23.444324 29444 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4a91eafc5ecf45b09f60d712884b7c79"
dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:35264
I20240502 20:46:23.444486 29444 raft_consensus.cc:595] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Received request to transfer leadership
I20240502 20:46:23.457598 29038 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:23.462306 31867 tablet_service.cc:1467] Processing CreateTablet for tablet eed562e3d88b4fb49362ed907fc191b9 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c4c16be8b1fd44208f8d28eb74b5b07b]), partition=
I20240502 20:46:23.462389 31605 tablet_service.cc:1467] Processing CreateTablet for tablet eed562e3d88b4fb49362ed907fc191b9 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c4c16be8b1fd44208f8d28eb74b5b07b]), partition=
I20240502 20:46:23.462594 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eed562e3d88b4fb49362ed907fc191b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:23.462671 31605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eed562e3d88b4fb49362ed907fc191b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:23.464562 3773 tablet_bootstrap.cc:492] T eed562e3d88b4fb49362ed907fc191b9 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:23.465234 3480 raft_consensus.cc:979] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6: : Instructing follower 7796ea3fba294d17a34386873969431d to start an election
I20240502 20:46:23.465320 3480 raft_consensus.cc:1067] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Signalling peer 7796ea3fba294d17a34386873969431d to start an election
I20240502 20:46:23.465402 3774 tablet_bootstrap.cc:492] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:23.465442 31735 tablet_service.cc:1467] Processing CreateTablet for tablet eed562e3d88b4fb49362ed907fc191b9 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c4c16be8b1fd44208f8d28eb74b5b07b]), partition=
I20240502 20:46:23.465675 31735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eed562e3d88b4fb49362ed907fc191b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:23.465909 3773 tablet_bootstrap.cc:654] T eed562e3d88b4fb49362ed907fc191b9 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:23.465975 29175 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4a91eafc5ecf45b09f60d712884b7c79"
dest_uuid: "7796ea3fba294d17a34386873969431d"
from {username='slave'} at 127.22.45.67:45133
I20240502 20:46:23.466122 29175 raft_consensus.cc:482] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:23.466171 29175 raft_consensus.cc:3037] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:23.467180 29175 raft_consensus.cc:504] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:23.467657 29175 leader_election.cc:288] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 2 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:23.467938 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a91eafc5ecf45b09f60d712884b7c79" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:23.468056 29444 raft_consensus.cc:3032] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:23.468055 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a91eafc5ecf45b09f60d712884b7c79" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:23.468098 29444 raft_consensus.cc:729] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:23.468149 31626 raft_consensus.cc:3037] T 4a91eafc5ecf45b09f60d712884b7c79 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:23.468278 3774 tablet_bootstrap.cc:654] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:23.468245 29444 consensus_queue.cc:260] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:23.468322 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a91eafc5ecf45b09f60d712884b7c79" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:23.468433 29315 raft_consensus.cc:3037] T 4a91eafc5ecf45b09f60d712884b7c79 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:23.468438 29444 raft_consensus.cc:3037] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:23.469033 31626 raft_consensus.cc:2454] T 4a91eafc5ecf45b09f60d712884b7c79 P 315915efcf2b413ca7f9d874338e1b2d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 2.
I20240502 20:46:23.469369 29444 raft_consensus.cc:2454] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 2.
I20240502 20:46:23.469393 3774 tablet_bootstrap.cc:492] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:23.469431 29315 raft_consensus.cc:2454] T 4a91eafc5ecf45b09f60d712884b7c79 P 09258ac095664fa99caf0707262cde13 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 2.
I20240502 20:46:23.469470 3774 ts_tablet_manager.cc:1367] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:23.469622 3773 tablet_bootstrap.cc:492] T eed562e3d88b4fb49362ed907fc191b9 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:23.469748 3773 ts_tablet_manager.cc:1367] T eed562e3d88b4fb49362ed907fc191b9 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 20:46:23.470081 3774 raft_consensus.cc:348] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:23.470178 3774 raft_consensus.cc:374] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:23.470214 3774 raft_consensus.cc:729] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:23.470188 3773 raft_consensus.cc:348] T eed562e3d88b4fb49362ed907fc191b9 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:23.470295 3773 raft_consensus.cc:374] T eed562e3d88b4fb49362ed907fc191b9 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:23.470329 3773 raft_consensus.cc:729] T eed562e3d88b4fb49362ed907fc191b9 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:23.470355 3774 consensus_queue.cc:260] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:23.470535 3773 consensus_queue.cc:260] T eed562e3d88b4fb49362ed907fc191b9 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:23.470682 3774 ts_tablet_manager.cc:1398] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:23.471380 3773 ts_tablet_manager.cc:1398] T eed562e3d88b4fb49362ed907fc191b9 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:23.471681 3776 tablet_bootstrap.cc:492] T eed562e3d88b4fb49362ed907fc191b9 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:23.472352 29119 leader_election.cc:302] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:23.472685 3642 raft_consensus.cc:2781] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:23.472779 3642 raft_consensus.cc:686] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d [term 2 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:23.472931 3642 consensus_queue.cc:237] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:23.473157 3776 tablet_bootstrap.cc:654] T eed562e3d88b4fb49362ed907fc191b9 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:23.475328 29037 catalog_manager.cc:5484] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 1 to 2, leader changed from dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 2 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:23.476589 3776 tablet_bootstrap.cc:492] T eed562e3d88b4fb49362ed907fc191b9 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:23.478386 3776 ts_tablet_manager.cc:1367] T eed562e3d88b4fb49362ed907fc191b9 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 20:46:23.478868 3776 raft_consensus.cc:348] T eed562e3d88b4fb49362ed907fc191b9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:23.478981 3776 raft_consensus.cc:374] T eed562e3d88b4fb49362ed907fc191b9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:23.479024 3776 raft_consensus.cc:729] T eed562e3d88b4fb49362ed907fc191b9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:23.479171 3776 consensus_queue.cc:260] T eed562e3d88b4fb49362ed907fc191b9 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:23.479493 3776 ts_tablet_manager.cc:1398] T eed562e3d88b4fb49362ed907fc191b9 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:23.518518 3810 ts_tablet_manager.cc:902] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6: Initiating tablet copy from peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:23.519387 3810 tablet_copy_client.cc:308] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.70:46751
I20240502 20:46:23.520061 31646 tablet_copy_service.cc:140] P 315915efcf2b413ca7f9d874338e1b2d: Received BeginTabletCopySession request for tablet 90bab3d11d424b3abcdc74d5da51cbee from peer dbcdc6ac635b44b185c3eba63f66e9d6 ({username='slave'} at 127.22.45.67:57269)
I20240502 20:46:23.520181 31646 tablet_copy_service.cc:161] P 315915efcf2b413ca7f9d874338e1b2d: Beginning new tablet copy session on tablet 90bab3d11d424b3abcdc74d5da51cbee from peer dbcdc6ac635b44b185c3eba63f66e9d6 at {username='slave'} at 127.22.45.67:57269: session id = dbcdc6ac635b44b185c3eba63f66e9d6-90bab3d11d424b3abcdc74d5da51cbee
I20240502 20:46:23.521198 31646 tablet_copy_source_session.cc:205] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:23.522274 3810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90bab3d11d424b3abcdc74d5da51cbee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:23.525369 3810 tablet_copy_client.cc:783] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:23.525552 3810 tablet_copy_client.cc:647] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:23.527555 3810 tablet_copy_client.cc:523] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:23.529762 3810 tablet_bootstrap.cc:492] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:23.534747 3810 tablet_bootstrap.cc:492] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:23.534986 3810 tablet_bootstrap.cc:492] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap complete.
I20240502 20:46:23.535140 3810 ts_tablet_manager.cc:1367] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240502 20:46:23.535542 3810 raft_consensus.cc:348] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:23.535650 3810 raft_consensus.cc:729] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:23.535764 3810 consensus_queue.cc:260] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:23.536021 3810 ts_tablet_manager.cc:1398] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:23.536513 31646 tablet_copy_service.cc:342] P 315915efcf2b413ca7f9d874338e1b2d: Request end of tablet copy session dbcdc6ac635b44b185c3eba63f66e9d6-90bab3d11d424b3abcdc74d5da51cbee received from {username='slave'} at 127.22.45.67:57269
I20240502 20:46:23.536594 31646 tablet_copy_service.cc:433] P 315915efcf2b413ca7f9d874338e1b2d: ending tablet copy session dbcdc6ac635b44b185c3eba63f66e9d6-90bab3d11d424b3abcdc74d5da51cbee on tablet 90bab3d11d424b3abcdc74d5da51cbee with peer dbcdc6ac635b44b185c3eba63f66e9d6
I20240502 20:46:23.618566 3400 raft_consensus.cc:482] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:23.618672 3400 raft_consensus.cc:504] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:23.619002 3400 leader_election.cc:288] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:23.619335 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eed562e3d88b4fb49362ed907fc191b9" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:23.619360 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eed562e3d88b4fb49362ed907fc191b9" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:23.619459 31756 raft_consensus.cc:2454] T eed562e3d88b4fb49362ed907fc191b9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:23.619489 31626 raft_consensus.cc:2454] T eed562e3d88b4fb49362ed907fc191b9 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:23.619694 31821 leader_election.cc:302] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e6b60803d0764d228c863e2da65e3691, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:23.619899 3400 raft_consensus.cc:2781] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:23.619951 3400 raft_consensus.cc:482] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:23.619987 3400 raft_consensus.cc:3037] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:23.620947 3400 raft_consensus.cc:504] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:23.621387 3400 leader_election.cc:288] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:23.621547 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eed562e3d88b4fb49362ed907fc191b9" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:23.621671 31626 raft_consensus.cc:3037] T eed562e3d88b4fb49362ed907fc191b9 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:23.621666 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eed562e3d88b4fb49362ed907fc191b9" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:23.621760 31757 raft_consensus.cc:3037] T eed562e3d88b4fb49362ed907fc191b9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:23.622587 31626 raft_consensus.cc:2454] T eed562e3d88b4fb49362ed907fc191b9 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:23.622684 31757 raft_consensus.cc:2454] T eed562e3d88b4fb49362ed907fc191b9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:23.622902 31822 leader_election.cc:302] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:23.623049 3400 raft_consensus.cc:2781] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:23.623340 3400 raft_consensus.cc:686] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Becoming Leader. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Running, Role: LEADER
I20240502 20:46:23.623495 3400 consensus_queue.cc:237] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:23.625449 29041 catalog_manager.cc:5484] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6b60803d0764d228c863e2da65e3691 (127.22.45.72). New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:23.654856 31474 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:23.655581 31605 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:23.656489 29295 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:23.668812 31735 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:23.669297 31867 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:23.671093 29163 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:23.673774 29424 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:23.687181 3400 consensus_queue.cc:1035] T 4aa8ea08d2b247b884cc14437d1441a8 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:23.690590 29557 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:23.692415 3828 ts_tablet_manager.cc:902] T 7fd046a0f0004e5fa6e0825cb626c2ff P e7ab6b9053664ee5aa71c7978e26fe6a: Initiating tablet copy from peer 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:23.692991 3828 tablet_copy_client.cc:308] T 7fd046a0f0004e5fa6e0825cb626c2ff P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.65:43267
I20240502 20:46:23.695456 29202 tablet_copy_service.cc:140] P 7796ea3fba294d17a34386873969431d: Received BeginTabletCopySession request for tablet 7fd046a0f0004e5fa6e0825cb626c2ff from peer e7ab6b9053664ee5aa71c7978e26fe6a ({username='slave'} at 127.22.45.71:50917)
I20240502 20:46:23.695561 29202 tablet_copy_service.cc:161] P 7796ea3fba294d17a34386873969431d: Beginning new tablet copy session on tablet 7fd046a0f0004e5fa6e0825cb626c2ff from peer e7ab6b9053664ee5aa71c7978e26fe6a at {username='slave'} at 127.22.45.71:50917: session id = e7ab6b9053664ee5aa71c7978e26fe6a-7fd046a0f0004e5fa6e0825cb626c2ff
I20240502 20:46:23.696499 29202 tablet_copy_source_session.cc:205] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:23.698036 3284 consensus_queue.cc:1035] T 4aa8ea08d2b247b884cc14437d1441a8 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:23.699685 3828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fd046a0f0004e5fa6e0825cb626c2ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:23.701972 3828 tablet_copy_client.cc:783] T 7fd046a0f0004e5fa6e0825cb626c2ff P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:23.702165 3828 tablet_copy_client.cc:647] T 7fd046a0f0004e5fa6e0825cb626c2ff P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:23.708984 3828 tablet_copy_client.cc:523] T 7fd046a0f0004e5fa6e0825cb626c2ff P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:23.709873 3828 tablet_bootstrap.cc:492] T 7fd046a0f0004e5fa6e0825cb626c2ff P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:23.729290 3828 tablet_bootstrap.cc:492] T 7fd046a0f0004e5fa6e0825cb626c2ff P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:23.729506 3828 tablet_bootstrap.cc:492] T 7fd046a0f0004e5fa6e0825cb626c2ff P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap complete.
I20240502 20:46:23.729660 3828 ts_tablet_manager.cc:1367] T 7fd046a0f0004e5fa6e0825cb626c2ff P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.020s user 0.017s sys 0.000s
I20240502 20:46:23.730003 3828 raft_consensus.cc:348] T 7fd046a0f0004e5fa6e0825cb626c2ff P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:23.730139 3828 raft_consensus.cc:729] T 7fd046a0f0004e5fa6e0825cb626c2ff P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:23.730288 3828 consensus_queue.cc:260] T 7fd046a0f0004e5fa6e0825cb626c2ff P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:23.730723 3828 ts_tablet_manager.cc:1398] T 7fd046a0f0004e5fa6e0825cb626c2ff P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 20:46:23.731405 29202 tablet_copy_service.cc:342] P 7796ea3fba294d17a34386873969431d: Request end of tablet copy session e7ab6b9053664ee5aa71c7978e26fe6a-7fd046a0f0004e5fa6e0825cb626c2ff received from {username='slave'} at 127.22.45.71:50917
I20240502 20:46:23.731478 29202 tablet_copy_service.cc:433] P 7796ea3fba294d17a34386873969431d: ending tablet copy session e7ab6b9053664ee5aa71c7978e26fe6a-7fd046a0f0004e5fa6e0825cb626c2ff on tablet 7fd046a0f0004e5fa6e0825cb626c2ff with peer e7ab6b9053664ee5aa71c7978e26fe6a
I20240502 20:46:23.731721 29037 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:23.735895 31626 raft_consensus.cc:1261] T eed562e3d88b4fb49362ed907fc191b9 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:23.736474 31755 raft_consensus.cc:1261] T eed562e3d88b4fb49362ed907fc191b9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:23.736871 3400 consensus_queue.cc:1035] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:23.737167 3284 consensus_queue.cc:1035] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:23.741176 3589 tablet.cc:1722] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:23.741458 3589 tablet.cc:1620] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:23.742255 3839 tablet.cc:1722] T eed562e3d88b4fb49362ed907fc191b9 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:23.742449 29037 catalog_manager.cc:5576] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751) does not have the latest schema for tablet eed562e3d88b4fb49362ed907fc191b9 (table rebalancer_dynamic_table [id=c4c16be8b1fd44208f8d28eb74b5b07b]). Expected version 1 got 0
I20240502 20:46:23.742672 3839 tablet.cc:1620] T eed562e3d88b4fb49362ed907fc191b9 P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:23.743405 29041 catalog_manager.cc:5576] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787) does not have the latest schema for tablet eed562e3d88b4fb49362ed907fc191b9 (table rebalancer_dynamic_table [id=c4c16be8b1fd44208f8d28eb74b5b07b]). Expected version 1 got 0
I20240502 20:46:23.746412 29040 catalog_manager.cc:5576] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751) does not have the latest schema for tablet eed562e3d88b4fb49362ed907fc191b9 (table rebalancer_dynamic_table [id=c4c16be8b1fd44208f8d28eb74b5b07b]). Expected version 1 got 0
I20240502 20:46:23.746542 3840 tablet.cc:1722] T eed562e3d88b4fb49362ed907fc191b9 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:23.746816 3840 tablet.cc:1620] T eed562e3d88b4fb49362ed907fc191b9 P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:23.747260 29041 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=c4c16be8b1fd44208f8d28eb74b5b07b] alter complete (version 1)
I20240502 20:46:23.750306 29041 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_30"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:23.756726 31474 tablet_service.cc:1467] Processing CreateTablet for tablet 75ace61b6538462b9bd5c0820eeacca9 (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=ff32eaad49184ef9bdfa32309107f0ac]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:23.757007 31474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75ace61b6538462b9bd5c0820eeacca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:23.757190 31475 tablet_service.cc:1467] Processing CreateTablet for tablet 2487c67d0f104af0b7d97fcd99bed431 (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=ff32eaad49184ef9bdfa32309107f0ac]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:23.757433 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2487c67d0f104af0b7d97fcd99bed431. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:23.757711 31735 tablet_service.cc:1467] Processing CreateTablet for tablet 75ace61b6538462b9bd5c0820eeacca9 (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=ff32eaad49184ef9bdfa32309107f0ac]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:23.757936 31735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75ace61b6538462b9bd5c0820eeacca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:23.759315 31465 tablet_service.cc:1467] Processing CreateTablet for tablet 3eb48ef3d5404e5e9c302b904b5df2cd (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=ff32eaad49184ef9bdfa32309107f0ac]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:23.759521 31465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3eb48ef3d5404e5e9c302b904b5df2cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:23.759651 29557 tablet_service.cc:1467] Processing CreateTablet for tablet 2487c67d0f104af0b7d97fcd99bed431 (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=ff32eaad49184ef9bdfa32309107f0ac]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:23.759868 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2487c67d0f104af0b7d97fcd99bed431. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:23.759924 3776 tablet_bootstrap.cc:492] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:23.760155 29163 tablet_service.cc:1467] Processing CreateTablet for tablet 2487c67d0f104af0b7d97fcd99bed431 (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=ff32eaad49184ef9bdfa32309107f0ac]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:23.760404 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2487c67d0f104af0b7d97fcd99bed431. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:23.760766 3776 tablet_bootstrap.cc:654] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:23.762059 3776 tablet_bootstrap.cc:492] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:23.762132 3776 ts_tablet_manager.cc:1367] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:23.762235 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 75ace61b6538462b9bd5c0820eeacca9 (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=ff32eaad49184ef9bdfa32309107f0ac]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:23.762502 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75ace61b6538462b9bd5c0820eeacca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:23.762914 3776 raft_consensus.cc:348] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:23.762976 3845 tablet_bootstrap.cc:492] T 75ace61b6538462b9bd5c0820eeacca9 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:23.763000 3776 raft_consensus.cc:374] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:23.763025 3776 raft_consensus.cc:729] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:23.763126 3776 consensus_queue.cc:260] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:23.763368 3776 ts_tablet_manager.cc:1398] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:23.763911 3845 tablet_bootstrap.cc:654] T 75ace61b6538462b9bd5c0820eeacca9 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:23.764554 3774 tablet_bootstrap.cc:492] T 75ace61b6538462b9bd5c0820eeacca9 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:23.765410 3847 tablet_bootstrap.cc:492] T 2487c67d0f104af0b7d97fcd99bed431 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:23.765430 3774 tablet_bootstrap.cc:654] T 75ace61b6538462b9bd5c0820eeacca9 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:23.765525 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 3eb48ef3d5404e5e9c302b904b5df2cd (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=ff32eaad49184ef9bdfa32309107f0ac]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:23.765734 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3eb48ef3d5404e5e9c302b904b5df2cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:23.766258 31605 tablet_service.cc:1467] Processing CreateTablet for tablet 3eb48ef3d5404e5e9c302b904b5df2cd (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=ff32eaad49184ef9bdfa32309107f0ac]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:23.766323 3847 tablet_bootstrap.cc:654] T 2487c67d0f104af0b7d97fcd99bed431 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:23.766515 31605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3eb48ef3d5404e5e9c302b904b5df2cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:23.766968 3848 tablet_bootstrap.cc:492] T 2487c67d0f104af0b7d97fcd99bed431 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:23.767395 3847 tablet_bootstrap.cc:492] T 2487c67d0f104af0b7d97fcd99bed431 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:23.767480 3847 ts_tablet_manager.cc:1367] T 2487c67d0f104af0b7d97fcd99bed431 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:23.767874 3847 raft_consensus.cc:348] T 2487c67d0f104af0b7d97fcd99bed431 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:23.767925 3848 tablet_bootstrap.cc:654] T 2487c67d0f104af0b7d97fcd99bed431 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:23.767951 3847 raft_consensus.cc:374] T 2487c67d0f104af0b7d97fcd99bed431 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:23.767969 3847 raft_consensus.cc:729] T 2487c67d0f104af0b7d97fcd99bed431 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:23.768049 3847 consensus_queue.cc:260] T 2487c67d0f104af0b7d97fcd99bed431 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:23.768270 3847 ts_tablet_manager.cc:1398] T 2487c67d0f104af0b7d97fcd99bed431 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:23.768677 3773 tablet_bootstrap.cc:492] T 3eb48ef3d5404e5e9c302b904b5df2cd P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:23.769030 3848 tablet_bootstrap.cc:492] T 2487c67d0f104af0b7d97fcd99bed431 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:23.769102 3848 ts_tablet_manager.cc:1367] T 2487c67d0f104af0b7d97fcd99bed431 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:23.769352 3773 tablet_bootstrap.cc:654] T 3eb48ef3d5404e5e9c302b904b5df2cd P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:23.769498 3848 raft_consensus.cc:348] T 2487c67d0f104af0b7d97fcd99bed431 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:23.769598 3848 raft_consensus.cc:374] T 2487c67d0f104af0b7d97fcd99bed431 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:23.769644 3848 raft_consensus.cc:729] T 2487c67d0f104af0b7d97fcd99bed431 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:23.769759 3848 consensus_queue.cc:260] T 2487c67d0f104af0b7d97fcd99bed431 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:23.770790 3774 tablet_bootstrap.cc:492] T 75ace61b6538462b9bd5c0820eeacca9 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:23.770867 3774 ts_tablet_manager.cc:1367] T 75ace61b6538462b9bd5c0820eeacca9 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 20:46:23.771245 3848 ts_tablet_manager.cc:1398] T 2487c67d0f104af0b7d97fcd99bed431 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:23.771296 3774 raft_consensus.cc:348] T 75ace61b6538462b9bd5c0820eeacca9 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:23.771396 3774 raft_consensus.cc:374] T 75ace61b6538462b9bd5c0820eeacca9 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:23.771428 3774 raft_consensus.cc:729] T 75ace61b6538462b9bd5c0820eeacca9 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:23.771540 3774 consensus_queue.cc:260] T 75ace61b6538462b9bd5c0820eeacca9 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:23.771800 3774 ts_tablet_manager.cc:1398] T 75ace61b6538462b9bd5c0820eeacca9 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:23.771950 3774 tablet_bootstrap.cc:492] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:23.771973 3845 tablet_bootstrap.cc:492] T 75ace61b6538462b9bd5c0820eeacca9 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:23.772049 3845 ts_tablet_manager.cc:1367] T 75ace61b6538462b9bd5c0820eeacca9 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 20:46:23.772485 3845 raft_consensus.cc:348] T 75ace61b6538462b9bd5c0820eeacca9 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:23.772598 3845 raft_consensus.cc:374] T 75ace61b6538462b9bd5c0820eeacca9 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:23.772634 3845 raft_consensus.cc:729] T 75ace61b6538462b9bd5c0820eeacca9 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:23.772742 3845 consensus_queue.cc:260] T 75ace61b6538462b9bd5c0820eeacca9 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:23.772815 3774 tablet_bootstrap.cc:654] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:23.772984 3845 ts_tablet_manager.cc:1398] T 75ace61b6538462b9bd5c0820eeacca9 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:23.773090 3845 tablet_bootstrap.cc:492] T 2487c67d0f104af0b7d97fcd99bed431 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:23.773825 3845 tablet_bootstrap.cc:654] T 2487c67d0f104af0b7d97fcd99bed431 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:23.773867 3774 tablet_bootstrap.cc:492] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:23.773938 3774 ts_tablet_manager.cc:1367] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:23.774348 3774 raft_consensus.cc:348] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:23.774506 3774 raft_consensus.cc:374] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:23.774578 3774 raft_consensus.cc:729] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:23.775010 3774 consensus_queue.cc:260] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:23.775507 3774 ts_tablet_manager.cc:1398] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:23.776100 3845 tablet_bootstrap.cc:492] T 2487c67d0f104af0b7d97fcd99bed431 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:23.776182 3845 ts_tablet_manager.cc:1367] T 2487c67d0f104af0b7d97fcd99bed431 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:23.776551 3845 raft_consensus.cc:348] T 2487c67d0f104af0b7d97fcd99bed431 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:23.776645 3845 raft_consensus.cc:374] T 2487c67d0f104af0b7d97fcd99bed431 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:23.776679 3845 raft_consensus.cc:729] T 2487c67d0f104af0b7d97fcd99bed431 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:23.776782 3845 consensus_queue.cc:260] T 2487c67d0f104af0b7d97fcd99bed431 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:23.777036 3845 ts_tablet_manager.cc:1398] T 2487c67d0f104af0b7d97fcd99bed431 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:23.777192 3845 tablet_bootstrap.cc:492] T 3eb48ef3d5404e5e9c302b904b5df2cd P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:23.778024 3845 tablet_bootstrap.cc:654] T 3eb48ef3d5404e5e9c302b904b5df2cd P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:23.779727 3845 tablet_bootstrap.cc:492] T 3eb48ef3d5404e5e9c302b904b5df2cd P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:23.779814 3845 ts_tablet_manager.cc:1367] T 3eb48ef3d5404e5e9c302b904b5df2cd P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:23.779829 3773 tablet_bootstrap.cc:492] T 3eb48ef3d5404e5e9c302b904b5df2cd P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:23.779907 3773 ts_tablet_manager.cc:1367] T 3eb48ef3d5404e5e9c302b904b5df2cd P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240502 20:46:23.780332 3845 raft_consensus.cc:348] T 3eb48ef3d5404e5e9c302b904b5df2cd P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:23.780326 3773 raft_consensus.cc:348] T 3eb48ef3d5404e5e9c302b904b5df2cd P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:23.780411 3845 raft_consensus.cc:374] T 3eb48ef3d5404e5e9c302b904b5df2cd P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:23.780422 3773 raft_consensus.cc:374] T 3eb48ef3d5404e5e9c302b904b5df2cd P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:23.780454 3773 raft_consensus.cc:729] T 3eb48ef3d5404e5e9c302b904b5df2cd P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:23.780568 3773 consensus_queue.cc:260] T 3eb48ef3d5404e5e9c302b904b5df2cd P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:23.780840 3773 ts_tablet_manager.cc:1398] T 3eb48ef3d5404e5e9c302b904b5df2cd P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:23.780438 3845 raft_consensus.cc:729] T 3eb48ef3d5404e5e9c302b904b5df2cd P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:23.781453 3845 consensus_queue.cc:260] T 3eb48ef3d5404e5e9c302b904b5df2cd P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:23.781729 3845 ts_tablet_manager.cc:1398] T 3eb48ef3d5404e5e9c302b904b5df2cd P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:23.850193 29040 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:23.856858 3589 tablet.cc:1722] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:23.857182 3589 tablet.cc:1620] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:23.857532 3840 tablet.cc:1722] T eed562e3d88b4fb49362ed907fc191b9 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:23.857841 3840 tablet.cc:1620] T eed562e3d88b4fb49362ed907fc191b9 P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:23.859494 3839 tablet.cc:1722] T eed562e3d88b4fb49362ed907fc191b9 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:23.859746 3839 tablet.cc:1620] T eed562e3d88b4fb49362ed907fc191b9 P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:23.864460 29039 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=c4c16be8b1fd44208f8d28eb74b5b07b] alter complete (version 2)
I20240502 20:46:23.895084 3290 raft_consensus.cc:482] T 2487c67d0f104af0b7d97fcd99bed431 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:23.895200 3290 raft_consensus.cc:504] T 2487c67d0f104af0b7d97fcd99bed431 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:23.895531 3290 leader_election.cc:288] T 2487c67d0f104af0b7d97fcd99bed431 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:23.895908 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2487c67d0f104af0b7d97fcd99bed431" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:23.896037 29175 raft_consensus.cc:2454] T 2487c67d0f104af0b7d97fcd99bed431 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:23.896018 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2487c67d0f104af0b7d97fcd99bed431" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:23.896135 31495 raft_consensus.cc:2454] T 2487c67d0f104af0b7d97fcd99bed431 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:23.896349 29513 leader_election.cc:302] T 2487c67d0f104af0b7d97fcd99bed431 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:23.896499 3290 raft_consensus.cc:2781] T 2487c67d0f104af0b7d97fcd99bed431 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:23.896553 3290 raft_consensus.cc:482] T 2487c67d0f104af0b7d97fcd99bed431 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:23.896589 3290 raft_consensus.cc:3037] T 2487c67d0f104af0b7d97fcd99bed431 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:23.897644 3290 raft_consensus.cc:504] T 2487c67d0f104af0b7d97fcd99bed431 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:23.897928 3290 leader_election.cc:288] T 2487c67d0f104af0b7d97fcd99bed431 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Requested vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:23.898186 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2487c67d0f104af0b7d97fcd99bed431" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:23.898298 31495 raft_consensus.cc:3037] T 2487c67d0f104af0b7d97fcd99bed431 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:23.898280 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2487c67d0f104af0b7d97fcd99bed431" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:23.898378 29175 raft_consensus.cc:3037] T 2487c67d0f104af0b7d97fcd99bed431 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:23.899149 29175 raft_consensus.cc:2454] T 2487c67d0f104af0b7d97fcd99bed431 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:23.899238 31495 raft_consensus.cc:2454] T 2487c67d0f104af0b7d97fcd99bed431 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:23.899469 29513 leader_election.cc:302] T 2487c67d0f104af0b7d97fcd99bed431 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:23.899729 3290 raft_consensus.cc:2781] T 2487c67d0f104af0b7d97fcd99bed431 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:23.899889 3290 raft_consensus.cc:686] T 2487c67d0f104af0b7d97fcd99bed431 P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Becoming Leader. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Running, Role: LEADER
I20240502 20:46:23.900031 3290 consensus_queue.cc:237] T 2487c67d0f104af0b7d97fcd99bed431 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:23.901849 29040 catalog_manager.cc:5484] T 2487c67d0f104af0b7d97fcd99bed431 P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: term changed from 0 to 1, leader changed from <none> to db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:23.916306 3400 raft_consensus.cc:482] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:23.916393 3400 raft_consensus.cc:504] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:23.916718 3400 leader_election.cc:288] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:23.917016 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3eb48ef3d5404e5e9c302b904b5df2cd" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:23.917049 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3eb48ef3d5404e5e9c302b904b5df2cd" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:23.917150 31495 raft_consensus.cc:2454] T 3eb48ef3d5404e5e9c302b904b5df2cd P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:23.917156 31626 raft_consensus.cc:2454] T 3eb48ef3d5404e5e9c302b904b5df2cd P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:23.917418 31823 leader_election.cc:302] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:23.917619 3400 raft_consensus.cc:2781] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:23.917678 3400 raft_consensus.cc:482] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:23.917716 3400 raft_consensus.cc:3037] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:23.918612 3400 raft_consensus.cc:504] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:23.918905 3400 leader_election.cc:288] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:23.919168 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3eb48ef3d5404e5e9c302b904b5df2cd" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:23.919184 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3eb48ef3d5404e5e9c302b904b5df2cd" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:23.919261 31495 raft_consensus.cc:3037] T 3eb48ef3d5404e5e9c302b904b5df2cd P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:23.919263 31626 raft_consensus.cc:3037] T 3eb48ef3d5404e5e9c302b904b5df2cd P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:23.920017 31495 raft_consensus.cc:2454] T 3eb48ef3d5404e5e9c302b904b5df2cd P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:23.920099 31626 raft_consensus.cc:2454] T 3eb48ef3d5404e5e9c302b904b5df2cd P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:23.920284 31823 leader_election.cc:302] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:23.920420 3400 raft_consensus.cc:2781] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:23.920496 3400 raft_consensus.cc:686] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Becoming Leader. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Running, Role: LEADER
I20240502 20:46:23.920619 3400 consensus_queue.cc:237] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:23.922499 29040 catalog_manager.cc:5484] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6b60803d0764d228c863e2da65e3691 (127.22.45.72). New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:23.942095 29313 raft_consensus.cc:1261] T 4a91eafc5ecf45b09f60d712884b7c79 P 09258ac095664fa99caf0707262cde13 [term 2 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:23.942515 3642 consensus_queue.cc:1035] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:23.949075 29444 raft_consensus.cc:1203] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 20:46:23.967216 29039 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:23.971753 3589 tablet.cc:1722] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:23.972218 3840 tablet.cc:1722] T eed562e3d88b4fb49362ed907fc191b9 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:23.972612 3839 tablet.cc:1722] T eed562e3d88b4fb49362ed907fc191b9 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:23.976503 29040 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=c4c16be8b1fd44208f8d28eb74b5b07b] alter complete (version 3)
I20240502 20:46:24.035409 31626 raft_consensus.cc:1261] T 4a91eafc5ecf45b09f60d712884b7c79 P 315915efcf2b413ca7f9d874338e1b2d [term 2 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:24.035780 3642 consensus_queue.cc:1035] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:24.038210 29444 raft_consensus.cc:1261] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 2 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:24.038599 1776 consensus_queue.cc:1035] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:24.077939 29040 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:24.078058 29040 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:24.080135 29040 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet eed562e3d88b4fb49362ed907fc191b9
I20240502 20:46:24.080708 31867 tablet_service.cc:1514] Processing DeleteTablet for tablet eed562e3d88b4fb49362ed907fc191b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:24 UTC) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:24.080718 31605 tablet_service.cc:1514] Processing DeleteTablet for tablet eed562e3d88b4fb49362ed907fc191b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:24 UTC) from {username='slave'} at 127.0.0.1:58346
I20240502 20:46:24.081036 3904 tablet_replica.cc:330] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:24.081291 3904 raft_consensus.cc:2229] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:24.081565 3904 raft_consensus.cc:2258] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:24.081650 31735 tablet_service.cc:1514] Processing DeleteTablet for tablet eed562e3d88b4fb49362ed907fc191b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:24 UTC) from {username='slave'} at 127.0.0.1:33864
I20240502 20:46:24.081944 3905 tablet_replica.cc:330] T eed562e3d88b4fb49362ed907fc191b9 P e7ab6b9053664ee5aa71c7978e26fe6a: stopping tablet replica
I20240502 20:46:24.082056 3905 raft_consensus.cc:2229] T eed562e3d88b4fb49362ed907fc191b9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:24.082151 3905 raft_consensus.cc:2258] T eed562e3d88b4fb49362ed907fc191b9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:24.082177 3904 ts_tablet_manager.cc:1875] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:24.082368 3903 tablet_replica.cc:330] T eed562e3d88b4fb49362ed907fc191b9 P 315915efcf2b413ca7f9d874338e1b2d: stopping tablet replica
I20240502 20:46:24.082562 3903 raft_consensus.cc:2229] T eed562e3d88b4fb49362ed907fc191b9 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:24.082680 3903 raft_consensus.cc:2258] T eed562e3d88b4fb49362ed907fc191b9 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:24.082707 3905 ts_tablet_manager.cc:1875] T eed562e3d88b4fb49362ed907fc191b9 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:24.083165 3903 ts_tablet_manager.cc:1875] T eed562e3d88b4fb49362ed907fc191b9 P 315915efcf2b413ca7f9d874338e1b2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:24.084563 3904 ts_tablet_manager.cc:1888] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:24.084641 3904 log.cc:1196] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/eed562e3d88b4fb49362ed907fc191b9
I20240502 20:46:24.084805 3903 ts_tablet_manager.cc:1888] T eed562e3d88b4fb49362ed907fc191b9 P 315915efcf2b413ca7f9d874338e1b2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:24.084868 3903 log.cc:1196] T eed562e3d88b4fb49362ed907fc191b9 P 315915efcf2b413ca7f9d874338e1b2d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal/wals/eed562e3d88b4fb49362ed907fc191b9
I20240502 20:46:24.084892 3904 ts_tablet_manager.cc:1909] T eed562e3d88b4fb49362ed907fc191b9 P e6b60803d0764d228c863e2da65e3691: Deleting consensus metadata
I20240502 20:46:24.085146 3903 ts_tablet_manager.cc:1909] T eed562e3d88b4fb49362ed907fc191b9 P 315915efcf2b413ca7f9d874338e1b2d: Deleting consensus metadata
I20240502 20:46:24.085561 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet eed562e3d88b4fb49362ed907fc191b9 (table rebalancer_dynamic_table_new_name [id=c4c16be8b1fd44208f8d28eb74b5b07b]) successfully deleted
I20240502 20:46:24.085718 29027 catalog_manager.cc:4830] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): tablet eed562e3d88b4fb49362ed907fc191b9 (table rebalancer_dynamic_table_new_name [id=c4c16be8b1fd44208f8d28eb74b5b07b]) successfully deleted
I20240502 20:46:24.085808 3905 ts_tablet_manager.cc:1888] T eed562e3d88b4fb49362ed907fc191b9 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:24.085912 3905 log.cc:1196] T eed562e3d88b4fb49362ed907fc191b9 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/wals/eed562e3d88b4fb49362ed907fc191b9
I20240502 20:46:24.086139 3905 ts_tablet_manager.cc:1909] T eed562e3d88b4fb49362ed907fc191b9 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting consensus metadata
I20240502 20:46:24.086731 29026 catalog_manager.cc:4830] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): tablet eed562e3d88b4fb49362ed907fc191b9 (table rebalancer_dynamic_table_new_name [id=c4c16be8b1fd44208f8d28eb74b5b07b]) successfully deleted
I20240502 20:46:24.099617 3740 raft_consensus.cc:482] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:24.099709 3740 raft_consensus.cc:504] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:24.100029 3740 leader_election.cc:288] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:24.100294 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75ace61b6538462b9bd5c0820eeacca9" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:24.100338 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75ace61b6538462b9bd5c0820eeacca9" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:24.100428 31888 raft_consensus.cc:2454] T 75ace61b6538462b9bd5c0820eeacca9 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:24.100430 31495 raft_consensus.cc:2454] T 75ace61b6538462b9bd5c0820eeacca9 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:24.100698 31692 leader_election.cc:302] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:24.100859 3740 raft_consensus.cc:2781] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:24.100915 3740 raft_consensus.cc:482] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:24.100941 3740 raft_consensus.cc:3037] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:24.101656 3740 raft_consensus.cc:504] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:24.101948 3740 leader_election.cc:288] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Requested vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:24.102197 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75ace61b6538462b9bd5c0820eeacca9" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:24.102217 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75ace61b6538462b9bd5c0820eeacca9" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:24.102289 31495 raft_consensus.cc:3037] T 75ace61b6538462b9bd5c0820eeacca9 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:24.102312 31888 raft_consensus.cc:3037] T 75ace61b6538462b9bd5c0820eeacca9 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:24.103047 31888 raft_consensus.cc:2454] T 75ace61b6538462b9bd5c0820eeacca9 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:24.103060 31495 raft_consensus.cc:2454] T 75ace61b6538462b9bd5c0820eeacca9 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:24.103294 31691 leader_election.cc:302] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e6b60803d0764d228c863e2da65e3691, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:24.103479 3740 raft_consensus.cc:2781] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:24.103572 3740 raft_consensus.cc:686] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Becoming Leader. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Running, Role: LEADER
I20240502 20:46:24.103787 3740 consensus_queue.cc:237] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:24.105772 29034 catalog_manager.cc:5484] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: term changed from 0 to 1, leader changed from <none> to e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71). New cstate: current_term: 1 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:24.140898 31465 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:24.140914 31605 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:24.142246 29295 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:24.151007 31867 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:24.155083 29163 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:24.160373 29557 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:24.166769 29424 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:24.166913 31735 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:24.181525 29034 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:24.197726 31465 tablet_service.cc:1467] Processing CreateTablet for tablet 174f09a13531405f9ee8375ae67bec23 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=7d51666f5acf4a73b5a726ac4cdcc2f1]), partition=
I20240502 20:46:24.198254 31465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 174f09a13531405f9ee8375ae67bec23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:24.199680 31605 tablet_service.cc:1467] Processing CreateTablet for tablet 174f09a13531405f9ee8375ae67bec23 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=7d51666f5acf4a73b5a726ac4cdcc2f1]), partition=
I20240502 20:46:24.200511 3845 tablet_bootstrap.cc:492] T 174f09a13531405f9ee8375ae67bec23 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:24.201421 3845 tablet_bootstrap.cc:654] T 174f09a13531405f9ee8375ae67bec23 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:24.200325 31605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 174f09a13531405f9ee8375ae67bec23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:24.202286 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 174f09a13531405f9ee8375ae67bec23 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=7d51666f5acf4a73b5a726ac4cdcc2f1]), partition=
I20240502 20:46:24.202561 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 174f09a13531405f9ee8375ae67bec23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:24.204548 3774 tablet_bootstrap.cc:492] T 174f09a13531405f9ee8375ae67bec23 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:24.205415 3773 tablet_bootstrap.cc:492] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:24.207087 3774 tablet_bootstrap.cc:654] T 174f09a13531405f9ee8375ae67bec23 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:24.207748 3773 tablet_bootstrap.cc:654] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:24.209048 3774 tablet_bootstrap.cc:492] T 174f09a13531405f9ee8375ae67bec23 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:24.209136 3774 ts_tablet_manager.cc:1367] T 174f09a13531405f9ee8375ae67bec23 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 20:46:24.209762 3845 tablet_bootstrap.cc:492] T 174f09a13531405f9ee8375ae67bec23 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:24.209717 3774 raft_consensus.cc:348] T 174f09a13531405f9ee8375ae67bec23 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:24.209818 3774 raft_consensus.cc:374] T 174f09a13531405f9ee8375ae67bec23 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:24.209831 3845 ts_tablet_manager.cc:1367] T 174f09a13531405f9ee8375ae67bec23 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 20:46:24.209846 3774 raft_consensus.cc:729] T 174f09a13531405f9ee8375ae67bec23 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:24.209955 3774 consensus_queue.cc:260] T 174f09a13531405f9ee8375ae67bec23 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:24.210211 3774 ts_tablet_manager.cc:1398] T 174f09a13531405f9ee8375ae67bec23 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:24.210222 3845 raft_consensus.cc:348] T 174f09a13531405f9ee8375ae67bec23 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:24.210307 3845 raft_consensus.cc:374] T 174f09a13531405f9ee8375ae67bec23 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:24.210331 3845 raft_consensus.cc:729] T 174f09a13531405f9ee8375ae67bec23 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:24.210448 3845 consensus_queue.cc:260] T 174f09a13531405f9ee8375ae67bec23 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:24.210704 3845 ts_tablet_manager.cc:1398] T 174f09a13531405f9ee8375ae67bec23 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:24.222111 3773 tablet_bootstrap.cc:492] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:24.222198 3773 ts_tablet_manager.cc:1367] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 20:46:24.222645 3773 raft_consensus.cc:348] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:24.222757 3773 raft_consensus.cc:374] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:24.222792 3773 raft_consensus.cc:729] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:24.222903 3773 consensus_queue.cc:260] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:24.223198 3773 ts_tablet_manager.cc:1398] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:24.242571 31888 consensus_queue.cc:237] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:24.243778 29175 raft_consensus.cc:1261] T 6b675480162c4a9799db51ea4b6504a5 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:24.243831 29313 raft_consensus.cc:1261] T 6b675480162c4a9799db51ea4b6504a5 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:24.244177 3284 consensus_queue.cc:1035] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:24.244391 3898 consensus_queue.cc:1035] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:24.246038 3400 raft_consensus.cc:2932] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:24.246594 29175 raft_consensus.cc:2932] T 6b675480162c4a9799db51ea4b6504a5 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:24.246659 29313 raft_consensus.cc:2932] T 6b675480162c4a9799db51ea4b6504a5 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:24.249470 29040 catalog_manager.cc:5484] T 6b675480162c4a9799db51ea4b6504a5 P 09258ac095664fa99caf0707262cde13 reported cstate change: config changed from index 2 to 3, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) evicted. New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:24.252027 31465 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b675480162c4a9799db51ea4b6504a5 with delete_type TABLET_DATA_TOMBSTONED (TS 2a9c1af8cf814030bde56b67ca5c2fbe not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:40276
I20240502 20:46:24.252519 3945 tablet_replica.cc:330] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe: stopping tablet replica
I20240502 20:46:24.252812 3945 raft_consensus.cc:2229] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:24.252969 3945 raft_consensus.cc:2258] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:24.253401 3945 ts_tablet_manager.cc:1875] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:24.255971 3945 ts_tablet_manager.cc:1888] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 20:46:24.256078 3945 log.cc:1196] T 6b675480162c4a9799db51ea4b6504a5 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/wals/6b675480162c4a9799db51ea4b6504a5
I20240502 20:46:24.256719 29028 catalog_manager.cc:4830] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): tablet 6b675480162c4a9799db51ea4b6504a5 (table rebalancer_extra_table_0 [id=bec3bd33882044f38d797093171eeb1d]) successfully deleted
I20240502 20:46:24.293385 31755 raft_consensus.cc:1203] T 7fd046a0f0004e5fa6e0825cb626c2ff P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:24.304636 2289 raft_consensus.cc:482] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:24.304741 2289 raft_consensus.cc:504] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:24.305119 2289 leader_election.cc:288] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:24.305464 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "174f09a13531405f9ee8375ae67bec23" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:24.305594 31495 raft_consensus.cc:2454] T 174f09a13531405f9ee8375ae67bec23 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:24.305883 31561 leader_election.cc:302] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, 315915efcf2b413ca7f9d874338e1b2d; no voters:
I20240502 20:46:24.306064 2289 raft_consensus.cc:2781] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:24.306123 2289 raft_consensus.cc:482] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:24.306159 2289 raft_consensus.cc:3037] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:24.306555 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "174f09a13531405f9ee8375ae67bec23" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:24.306694 31888 raft_consensus.cc:2454] T 174f09a13531405f9ee8375ae67bec23 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:24.307729 2289 raft_consensus.cc:504] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:24.308063 2289 leader_election.cc:288] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Requested vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:24.308419 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "174f09a13531405f9ee8375ae67bec23" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:24.308521 31888 raft_consensus.cc:3037] T 174f09a13531405f9ee8375ae67bec23 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:24.308521 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "174f09a13531405f9ee8375ae67bec23" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:24.308609 31495 raft_consensus.cc:3037] T 174f09a13531405f9ee8375ae67bec23 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:24.309465 31495 raft_consensus.cc:2454] T 174f09a13531405f9ee8375ae67bec23 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:24.309465 31888 raft_consensus.cc:2454] T 174f09a13531405f9ee8375ae67bec23 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:24.309756 31560 leader_election.cc:302] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:24.309947 2289 raft_consensus.cc:2781] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:24.310096 2289 raft_consensus.cc:686] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Becoming Leader. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Running, Role: LEADER
I20240502 20:46:24.310235 2289 consensus_queue.cc:237] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:24.313395 29033 catalog_manager.cc:5484] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70). New cstate: current_term: 1 leader_uuid: "315915efcf2b413ca7f9d874338e1b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:24.340163 31465 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:24.342918 31605 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:24.350518 3894 consensus_queue.cc:1035] T 2487c67d0f104af0b7d97fcd99bed431 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:24.351001 29557 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:24.354853 29424 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:24.355458 29163 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:24.372905 31735 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:24.373456 29295 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:24.374639 31867 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:24.377175 3290 consensus_queue.cc:1035] T 2487c67d0f104af0b7d97fcd99bed431 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:24.407768 3898 consensus_queue.cc:1035] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:24.416175 3400 consensus_queue.cc:1035] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:24.430202 29038 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:24.435001 31495 raft_consensus.cc:1261] T 174f09a13531405f9ee8375ae67bec23 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Refusing update from remote peer 315915efcf2b413ca7f9d874338e1b2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:24.435236 31888 raft_consensus.cc:1261] T 174f09a13531405f9ee8375ae67bec23 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer 315915efcf2b413ca7f9d874338e1b2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:24.435736 3982 consensus_queue.cc:1035] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:24.437269 3982 consensus_queue.cc:1035] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:24.439020 29313 consensus_queue.cc:237] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:24.439361 29036 catalog_manager.cc:5576] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207) does not have the latest schema for tablet 174f09a13531405f9ee8375ae67bec23 (table rebalancer_dynamic_table [id=7d51666f5acf4a73b5a726ac4cdcc2f1]). Expected version 1 got 0
I20240502 20:46:24.439391 3840 tablet.cc:1722] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:24.439496 3264 mvcc.cc:205] Tried to move back new op lower bound from 7023340685040304128 to 7023340684536041472. Current Snapshot: MvccSnapshot[applied={T|T < 7023340685040304128}]
I20240502 20:46:24.439688 3840 tablet.cc:1620] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:24.440467 31626 raft_consensus.cc:1261] T d4a2c7b197f948a0b236d5e05e2749b9 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:24.440918 3531 consensus_queue.cc:1035] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:24.443356 4019 raft_consensus.cc:2932] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:24.445240 31626 raft_consensus.cc:2932] T d4a2c7b197f948a0b236d5e05e2749b9 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:24.446928 3589 tablet.cc:1722] T 174f09a13531405f9ee8375ae67bec23 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:24.447213 3589 tablet.cc:1620] T 174f09a13531405f9ee8375ae67bec23 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:24.447711 29034 catalog_manager.cc:5484] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 reported cstate change: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:24.448249 29038 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=7d51666f5acf4a73b5a726ac4cdcc2f1] alter complete (version 1)
I20240502 20:46:24.448390 29175 raft_consensus.cc:1261] T d4a2c7b197f948a0b236d5e05e2749b9 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:24.449684 4019 consensus_queue.cc:1035] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:24.450628 29036 catalog_manager.cc:5576] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887) does not have the latest schema for tablet 174f09a13531405f9ee8375ae67bec23 (table rebalancer_dynamic_table [id=7d51666f5acf4a73b5a726ac4cdcc2f1]). Expected version 1 got 0
I20240502 20:46:24.450830 29175 raft_consensus.cc:2932] T d4a2c7b197f948a0b236d5e05e2749b9 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:24.453305 29424 tablet_service.cc:1514] Processing DeleteTablet for tablet d4a2c7b197f948a0b236d5e05e2749b9 with delete_type TABLET_DATA_TOMBSTONED (TS dbcdc6ac635b44b185c3eba63f66e9d6 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:24.453545 29038 catalog_manager.cc:5576] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887) does not have the latest schema for tablet 174f09a13531405f9ee8375ae67bec23 (table rebalancer_dynamic_table [id=7d51666f5acf4a73b5a726ac4cdcc2f1]). Expected version 1 got 0
I20240502 20:46:24.455266 4028 tablet_replica.cc:330] T d4a2c7b197f948a0b236d5e05e2749b9 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:24.455459 4028 raft_consensus.cc:2229] T d4a2c7b197f948a0b236d5e05e2749b9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:24.455587 4021 tablet.cc:1722] T 174f09a13531405f9ee8375ae67bec23 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:24.456041 4021 tablet.cc:1620] T 174f09a13531405f9ee8375ae67bec23 P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:24.457247 4028 raft_consensus.cc:2258] T d4a2c7b197f948a0b236d5e05e2749b9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:24.457795 4028 ts_tablet_manager.cc:1875] T d4a2c7b197f948a0b236d5e05e2749b9 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:24.461266 4028 ts_tablet_manager.cc:1888] T d4a2c7b197f948a0b236d5e05e2749b9 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 20:46:24.461330 4028 log.cc:1196] T d4a2c7b197f948a0b236d5e05e2749b9 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/d4a2c7b197f948a0b236d5e05e2749b9
I20240502 20:46:24.462023 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet d4a2c7b197f948a0b236d5e05e2749b9 (table rebalance_test_table_5 [id=fd2f36435a614aa98d7f33e9646f33a3]) successfully deleted
I20240502 20:46:24.550333 29035 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:24.557318 3840 tablet.cc:1722] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:24.557655 3840 tablet.cc:1620] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:24.559830 4021 tablet.cc:1722] T 174f09a13531405f9ee8375ae67bec23 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:24.560133 4021 tablet.cc:1620] T 174f09a13531405f9ee8375ae67bec23 P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:24.560406 3589 tablet.cc:1722] T 174f09a13531405f9ee8375ae67bec23 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:24.560654 3589 tablet.cc:1620] T 174f09a13531405f9ee8375ae67bec23 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:24.567510 29035 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=7d51666f5acf4a73b5a726ac4cdcc2f1] alter complete (version 2)
I20240502 20:46:24.574548 29295 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:24.575146 3740 consensus_queue.cc:1035] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:24.578629 3289 consensus_queue.cc:1035] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:24.583811 31465 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:24.594645 31735 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:24.596937 29163 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:24.598565 29557 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:24.598811 31605 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:24.602603 29424 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:24.607059 31867 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:24.639972 29038 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_32"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:24.647295 31465 tablet_service.cc:1467] Processing CreateTablet for tablet d0538f41ea0e46edae87b8e4dfa995f0 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=a863244f6b74429ba652c196f42b4ee0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:24.647291 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 43fc3dcc2dd54a11becb8f16ec500df1 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=a863244f6b74429ba652c196f42b4ee0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:24.647572 31465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0538f41ea0e46edae87b8e4dfa995f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:24.647572 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43fc3dcc2dd54a11becb8f16ec500df1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:24.647572 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 43fc3dcc2dd54a11becb8f16ec500df1 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=a863244f6b74429ba652c196f42b4ee0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:24.647792 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43fc3dcc2dd54a11becb8f16ec500df1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:24.649660 3845 tablet_bootstrap.cc:492] T d0538f41ea0e46edae87b8e4dfa995f0 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:24.649963 31735 tablet_service.cc:1467] Processing CreateTablet for tablet 43fc3dcc2dd54a11becb8f16ec500df1 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=a863244f6b74429ba652c196f42b4ee0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:24.650208 31735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43fc3dcc2dd54a11becb8f16ec500df1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:24.650317 31737 tablet_service.cc:1467] Processing CreateTablet for tablet d0538f41ea0e46edae87b8e4dfa995f0 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=a863244f6b74429ba652c196f42b4ee0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:24.650583 3845 tablet_bootstrap.cc:654] T d0538f41ea0e46edae87b8e4dfa995f0 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:24.650683 31736 tablet_service.cc:1467] Processing CreateTablet for tablet 21eab607fca143b8a165d777b50238e1 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=a863244f6b74429ba652c196f42b4ee0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:24.650871 3774 tablet_bootstrap.cc:492] T 43fc3dcc2dd54a11becb8f16ec500df1 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:24.651743 3774 tablet_bootstrap.cc:654] T 43fc3dcc2dd54a11becb8f16ec500df1 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:24.651906 29557 tablet_service.cc:1467] Processing CreateTablet for tablet 21eab607fca143b8a165d777b50238e1 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=a863244f6b74429ba652c196f42b4ee0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:24.652009 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 21eab607fca143b8a165d777b50238e1 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=a863244f6b74429ba652c196f42b4ee0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:24.652166 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21eab607fca143b8a165d777b50238e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:24.652212 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21eab607fca143b8a165d777b50238e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:24.652520 31605 tablet_service.cc:1467] Processing CreateTablet for tablet d0538f41ea0e46edae87b8e4dfa995f0 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=a863244f6b74429ba652c196f42b4ee0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:24.652753 31605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0538f41ea0e46edae87b8e4dfa995f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:24.654439 3845 tablet_bootstrap.cc:492] T d0538f41ea0e46edae87b8e4dfa995f0 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:24.654537 3845 ts_tablet_manager.cc:1367] T d0538f41ea0e46edae87b8e4dfa995f0 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240502 20:46:24.655155 3845 raft_consensus.cc:348] T d0538f41ea0e46edae87b8e4dfa995f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:24.655289 3773 tablet_bootstrap.cc:492] T d0538f41ea0e46edae87b8e4dfa995f0 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:24.655339 3845 raft_consensus.cc:374] T d0538f41ea0e46edae87b8e4dfa995f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:24.655423 3845 raft_consensus.cc:729] T d0538f41ea0e46edae87b8e4dfa995f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:24.655750 3845 consensus_queue.cc:260] T d0538f41ea0e46edae87b8e4dfa995f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:24.656414 3845 ts_tablet_manager.cc:1398] T d0538f41ea0e46edae87b8e4dfa995f0 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240502 20:46:24.657656 3773 tablet_bootstrap.cc:654] T d0538f41ea0e46edae87b8e4dfa995f0 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:24.659781 4108 tablet_bootstrap.cc:492] T 21eab607fca143b8a165d777b50238e1 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:24.660400 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0538f41ea0e46edae87b8e4dfa995f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:24.660897 4108 tablet_bootstrap.cc:654] T 21eab607fca143b8a165d777b50238e1 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:24.661170 31736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21eab607fca143b8a165d777b50238e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:24.662181 4108 tablet_bootstrap.cc:492] T 21eab607fca143b8a165d777b50238e1 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:24.662256 4108 ts_tablet_manager.cc:1367] T 21eab607fca143b8a165d777b50238e1 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:24.662838 4108 raft_consensus.cc:348] T 21eab607fca143b8a165d777b50238e1 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:24.662936 4108 raft_consensus.cc:374] T 21eab607fca143b8a165d777b50238e1 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:24.662966 4108 raft_consensus.cc:729] T 21eab607fca143b8a165d777b50238e1 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:24.663115 4108 consensus_queue.cc:260] T 21eab607fca143b8a165d777b50238e1 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:24.663568 4108 ts_tablet_manager.cc:1398] T 21eab607fca143b8a165d777b50238e1 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:24.664224 4110 tablet_bootstrap.cc:492] T d0538f41ea0e46edae87b8e4dfa995f0 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:24.665233 4110 tablet_bootstrap.cc:654] T d0538f41ea0e46edae87b8e4dfa995f0 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:24.666275 3773 tablet_bootstrap.cc:492] T d0538f41ea0e46edae87b8e4dfa995f0 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:24.666359 3773 ts_tablet_manager.cc:1367] T d0538f41ea0e46edae87b8e4dfa995f0 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20240502 20:46:24.666801 3773 raft_consensus.cc:348] T d0538f41ea0e46edae87b8e4dfa995f0 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:24.667304 3773 raft_consensus.cc:374] T d0538f41ea0e46edae87b8e4dfa995f0 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:24.667394 3773 raft_consensus.cc:729] T d0538f41ea0e46edae87b8e4dfa995f0 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:24.667580 3773 consensus_queue.cc:260] T d0538f41ea0e46edae87b8e4dfa995f0 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:24.667948 4110 tablet_bootstrap.cc:492] T d0538f41ea0e46edae87b8e4dfa995f0 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:24.667980 3773 ts_tablet_manager.cc:1398] T d0538f41ea0e46edae87b8e4dfa995f0 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:24.668037 4110 ts_tablet_manager.cc:1367] T d0538f41ea0e46edae87b8e4dfa995f0 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:24.668437 4110 raft_consensus.cc:348] T d0538f41ea0e46edae87b8e4dfa995f0 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:24.668540 4110 raft_consensus.cc:374] T d0538f41ea0e46edae87b8e4dfa995f0 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:24.668604 4110 raft_consensus.cc:729] T d0538f41ea0e46edae87b8e4dfa995f0 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:24.668730 4110 consensus_queue.cc:260] T d0538f41ea0e46edae87b8e4dfa995f0 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:24.669005 4110 ts_tablet_manager.cc:1398] T d0538f41ea0e46edae87b8e4dfa995f0 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:24.669217 4110 tablet_bootstrap.cc:492] T 21eab607fca143b8a165d777b50238e1 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:24.669840 4104 tablet_bootstrap.cc:492] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:24.670048 4110 tablet_bootstrap.cc:654] T 21eab607fca143b8a165d777b50238e1 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:24.671177 3774 tablet_bootstrap.cc:492] T 43fc3dcc2dd54a11becb8f16ec500df1 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:24.671257 3774 ts_tablet_manager.cc:1367] T 43fc3dcc2dd54a11becb8f16ec500df1 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20240502 20:46:24.671656 3774 raft_consensus.cc:348] T 43fc3dcc2dd54a11becb8f16ec500df1 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:24.671746 3774 raft_consensus.cc:374] T 43fc3dcc2dd54a11becb8f16ec500df1 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:24.671773 3774 raft_consensus.cc:729] T 43fc3dcc2dd54a11becb8f16ec500df1 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:24.671885 3774 consensus_queue.cc:260] T 43fc3dcc2dd54a11becb8f16ec500df1 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:24.672103 3774 ts_tablet_manager.cc:1398] T 43fc3dcc2dd54a11becb8f16ec500df1 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:24.672267 3774 tablet_bootstrap.cc:492] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:24.672271 29033 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:24.673080 3774 tablet_bootstrap.cc:654] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:24.674636 3774 tablet_bootstrap.cc:492] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:24.674737 3774 ts_tablet_manager.cc:1367] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:24.675141 3774 raft_consensus.cc:348] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:24.675271 3774 raft_consensus.cc:374] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:24.675276 4104 tablet_bootstrap.cc:654] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:24.675326 3774 raft_consensus.cc:729] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:24.675496 3774 consensus_queue.cc:260] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:24.675837 3774 ts_tablet_manager.cc:1398] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:24.677652 3840 tablet.cc:1722] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:24.679431 4021 tablet.cc:1722] T 174f09a13531405f9ee8375ae67bec23 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:24.680403 3589 tablet.cc:1722] T 174f09a13531405f9ee8375ae67bec23 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:24.684947 29033 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=7d51666f5acf4a73b5a726ac4cdcc2f1] alter complete (version 3)
I20240502 20:46:24.686139 4110 tablet_bootstrap.cc:492] T 21eab607fca143b8a165d777b50238e1 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:24.686231 4104 tablet_bootstrap.cc:492] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:24.686273 4110 ts_tablet_manager.cc:1367] T 21eab607fca143b8a165d777b50238e1 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 20:46:24.686326 4104 ts_tablet_manager.cc:1367] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 20:46:24.686715 4110 raft_consensus.cc:348] T 21eab607fca143b8a165d777b50238e1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:24.686803 4110 raft_consensus.cc:374] T 21eab607fca143b8a165d777b50238e1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:24.686755 4104 raft_consensus.cc:348] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:24.686825 4110 raft_consensus.cc:729] T 21eab607fca143b8a165d777b50238e1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:24.686836 4104 raft_consensus.cc:374] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:24.686856 4104 raft_consensus.cc:729] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:24.686929 4110 consensus_queue.cc:260] T 21eab607fca143b8a165d777b50238e1 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:24.686959 4104 consensus_queue.cc:260] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:24.687183 4110 ts_tablet_manager.cc:1398] T 21eab607fca143b8a165d777b50238e1 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:24.687216 4104 ts_tablet_manager.cc:1398] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:24.687346 4110 tablet_bootstrap.cc:492] T 43fc3dcc2dd54a11becb8f16ec500df1 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:24.688270 4110 tablet_bootstrap.cc:654] T 43fc3dcc2dd54a11becb8f16ec500df1 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:24.689185 4110 tablet_bootstrap.cc:492] T 43fc3dcc2dd54a11becb8f16ec500df1 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:24.689260 4110 ts_tablet_manager.cc:1367] T 43fc3dcc2dd54a11becb8f16ec500df1 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:24.689651 4110 raft_consensus.cc:348] T 43fc3dcc2dd54a11becb8f16ec500df1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:24.689738 4110 raft_consensus.cc:374] T 43fc3dcc2dd54a11becb8f16ec500df1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:24.689767 4110 raft_consensus.cc:729] T 43fc3dcc2dd54a11becb8f16ec500df1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:24.689865 4110 consensus_queue.cc:260] T 43fc3dcc2dd54a11becb8f16ec500df1 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:24.690099 4110 ts_tablet_manager.cc:1398] T 43fc3dcc2dd54a11becb8f16ec500df1 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:24.693683 3898 raft_consensus.cc:482] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:24.693764 3898 raft_consensus.cc:504] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:24.694089 3898 leader_election.cc:288] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:24.694330 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21eab607fca143b8a165d777b50238e1" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:24.694442 31755 raft_consensus.cc:2454] T 21eab607fca143b8a165d777b50238e1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:24.694413 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21eab607fca143b8a165d777b50238e1" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:24.694587 29577 raft_consensus.cc:2454] T 21eab607fca143b8a165d777b50238e1 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:24.694742 31821 leader_election.cc:302] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e6b60803d0764d228c863e2da65e3691, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:24.694901 3898 raft_consensus.cc:2781] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:24.694945 3898 raft_consensus.cc:482] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:24.694976 3898 raft_consensus.cc:3037] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:24.695933 3898 raft_consensus.cc:504] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:24.696244 3898 leader_election.cc:288] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Requested vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:24.696542 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21eab607fca143b8a165d777b50238e1" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:24.696542 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21eab607fca143b8a165d777b50238e1" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:24.696645 31755 raft_consensus.cc:3037] T 21eab607fca143b8a165d777b50238e1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:24.696645 29577 raft_consensus.cc:3037] T 21eab607fca143b8a165d777b50238e1 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:24.697510 31755 raft_consensus.cc:2454] T 21eab607fca143b8a165d777b50238e1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:24.697506 29577 raft_consensus.cc:2454] T 21eab607fca143b8a165d777b50238e1 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:24.697757 31822 leader_election.cc:302] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:24.698165 3898 raft_consensus.cc:2781] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:24.698239 3898 raft_consensus.cc:686] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Becoming Leader. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Running, Role: LEADER
I20240502 20:46:24.698343 3898 consensus_queue.cc:237] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:24.700053 29038 catalog_manager.cc:5484] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6b60803d0764d228c863e2da65e3691 (127.22.45.72). New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:24.708935 29175 consensus_queue.cc:237] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:24.710033 31755 raft_consensus.cc:1261] T 7fd046a0f0004e5fa6e0825cb626c2ff P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:24.710142 29444 raft_consensus.cc:1261] T 7fd046a0f0004e5fa6e0825cb626c2ff P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:24.710516 4135 consensus_queue.cc:1035] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:24.710645 4122 consensus_queue.cc:1035] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:24.712359 4135 raft_consensus.cc:2932] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:24.712679 29444 raft_consensus.cc:2932] T 7fd046a0f0004e5fa6e0825cb626c2ff P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:24.715796 29035 catalog_manager.cc:5484] T 7fd046a0f0004e5fa6e0825cb626c2ff P 7796ea3fba294d17a34386873969431d reported cstate change: config changed from index 4 to 5, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) evicted. New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:24.717914 29295 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fd046a0f0004e5fa6e0825cb626c2ff with delete_type TABLET_DATA_TOMBSTONED (TS 09258ac095664fa99caf0707262cde13 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:24.718226 4143 tablet_replica.cc:330] T 7fd046a0f0004e5fa6e0825cb626c2ff P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:24.718412 4143 raft_consensus.cc:2229] T 7fd046a0f0004e5fa6e0825cb626c2ff P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:24.718585 4143 raft_consensus.cc:2258] T 7fd046a0f0004e5fa6e0825cb626c2ff P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:24.718995 4143 ts_tablet_manager.cc:1875] T 7fd046a0f0004e5fa6e0825cb626c2ff P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:24.720713 4143 ts_tablet_manager.cc:1888] T 7fd046a0f0004e5fa6e0825cb626c2ff P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 20:46:24.720814 4143 log.cc:1196] T 7fd046a0f0004e5fa6e0825cb626c2ff P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/7fd046a0f0004e5fa6e0825cb626c2ff
I20240502 20:46:24.721273 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet 7fd046a0f0004e5fa6e0825cb626c2ff (table rebalance_test_table_1 [id=8011864bab7b4506a89a8bbb019d244f]) successfully deleted
I20240502 20:46:24.723894 31755 raft_consensus.cc:2932] T 7fd046a0f0004e5fa6e0825cb626c2ff P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:24.787832 29042 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:24.788065 29042 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:24.790350 29042 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 174f09a13531405f9ee8375ae67bec23
I20240502 20:46:24.790867 31867 tablet_service.cc:1514] Processing DeleteTablet for tablet 174f09a13531405f9ee8375ae67bec23 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:24 UTC) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:24.790966 31605 tablet_service.cc:1514] Processing DeleteTablet for tablet 174f09a13531405f9ee8375ae67bec23 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:24 UTC) from {username='slave'} at 127.0.0.1:58346
I20240502 20:46:24.791288 4181 tablet_replica.cc:330] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d: stopping tablet replica
I20240502 20:46:24.791414 4180 tablet_replica.cc:330] T 174f09a13531405f9ee8375ae67bec23 P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:24.791390 31465 tablet_service.cc:1514] Processing DeleteTablet for tablet 174f09a13531405f9ee8375ae67bec23 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:24 UTC) from {username='slave'} at 127.0.0.1:40276
I20240502 20:46:24.791435 4181 raft_consensus.cc:2229] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:24.791569 4180 raft_consensus.cc:2229] T 174f09a13531405f9ee8375ae67bec23 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:24.791630 4181 raft_consensus.cc:2258] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:24.791712 4180 raft_consensus.cc:2258] T 174f09a13531405f9ee8375ae67bec23 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:24.791733 4019 raft_consensus.cc:482] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:24.791814 4019 raft_consensus.cc:504] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:24.792193 4019 leader_election.cc:288] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:24.792331 4181 ts_tablet_manager.cc:1875] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:24.792690 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43fc3dcc2dd54a11becb8f16ec500df1" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:24.792799 4180 ts_tablet_manager.cc:1875] T 174f09a13531405f9ee8375ae67bec23 P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:24.792814 31755 raft_consensus.cc:2454] T 43fc3dcc2dd54a11becb8f16ec500df1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:24.793118 29248 leader_election.cc:302] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:24.793566 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43fc3dcc2dd54a11becb8f16ec500df1" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:24.793680 31888 raft_consensus.cc:2454] T 43fc3dcc2dd54a11becb8f16ec500df1 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:24.794152 4019 raft_consensus.cc:2781] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:24.794214 4019 raft_consensus.cc:482] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:24.794248 4019 raft_consensus.cc:3037] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:24.795101 4180 ts_tablet_manager.cc:1888] T 174f09a13531405f9ee8375ae67bec23 P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:24.795315 4180 log.cc:1196] T 174f09a13531405f9ee8375ae67bec23 P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/174f09a13531405f9ee8375ae67bec23
I20240502 20:46:24.795281 4019 raft_consensus.cc:504] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:24.795574 4180 ts_tablet_manager.cc:1909] T 174f09a13531405f9ee8375ae67bec23 P e6b60803d0764d228c863e2da65e3691: Deleting consensus metadata
I20240502 20:46:24.795631 4019 leader_election.cc:288] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:24.796226 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43fc3dcc2dd54a11becb8f16ec500df1" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:24.796330 31755 raft_consensus.cc:3037] T 43fc3dcc2dd54a11becb8f16ec500df1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:24.796577 4181 ts_tablet_manager.cc:1888] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:24.796627 4181 log.cc:1196] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal/wals/174f09a13531405f9ee8375ae67bec23
I20240502 20:46:24.796795 4181 ts_tablet_manager.cc:1909] T 174f09a13531405f9ee8375ae67bec23 P 315915efcf2b413ca7f9d874338e1b2d: Deleting consensus metadata
I20240502 20:46:24.797345 31755 raft_consensus.cc:2454] T 43fc3dcc2dd54a11becb8f16ec500df1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:24.797353 29027 catalog_manager.cc:4830] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): tablet 174f09a13531405f9ee8375ae67bec23 (table rebalancer_dynamic_table_new_name [id=7d51666f5acf4a73b5a726ac4cdcc2f1]) successfully deleted
I20240502 20:46:24.797596 29248 leader_election.cc:302] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:24.797647 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet 174f09a13531405f9ee8375ae67bec23 (table rebalancer_dynamic_table_new_name [id=7d51666f5acf4a73b5a726ac4cdcc2f1]) successfully deleted
I20240502 20:46:24.797825 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43fc3dcc2dd54a11becb8f16ec500df1" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:24.797914 31888 raft_consensus.cc:3037] T 43fc3dcc2dd54a11becb8f16ec500df1 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:24.798056 4019 raft_consensus.cc:2781] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:24.798141 4019 raft_consensus.cc:686] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:24.798264 4019 consensus_queue.cc:237] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:24.798795 31888 raft_consensus.cc:2454] T 43fc3dcc2dd54a11becb8f16ec500df1 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:24.799773 4182 tablet_replica.cc:330] T 174f09a13531405f9ee8375ae67bec23 P 2a9c1af8cf814030bde56b67ca5c2fbe: stopping tablet replica
I20240502 20:46:24.799912 4182 raft_consensus.cc:2229] T 174f09a13531405f9ee8375ae67bec23 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:24.800058 4182 raft_consensus.cc:2258] T 174f09a13531405f9ee8375ae67bec23 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:24.800611 4182 ts_tablet_manager.cc:1875] T 174f09a13531405f9ee8375ae67bec23 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:24.800720 29035 catalog_manager.cc:5484] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:24.802888 4182 ts_tablet_manager.cc:1888] T 174f09a13531405f9ee8375ae67bec23 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:24.803009 4182 log.cc:1196] T 174f09a13531405f9ee8375ae67bec23 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/wals/174f09a13531405f9ee8375ae67bec23
I20240502 20:46:24.803454 4182 ts_tablet_manager.cc:1909] T 174f09a13531405f9ee8375ae67bec23 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting consensus metadata
I20240502 20:46:24.804009 29028 catalog_manager.cc:4830] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): tablet 174f09a13531405f9ee8375ae67bec23 (table rebalancer_dynamic_table_new_name [id=7d51666f5acf4a73b5a726ac4cdcc2f1]) successfully deleted
I20240502 20:46:24.816219 3829 raft_consensus.cc:482] T d0538f41ea0e46edae87b8e4dfa995f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:24.816299 3829 raft_consensus.cc:504] T d0538f41ea0e46edae87b8e4dfa995f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:24.816591 3829 leader_election.cc:288] T d0538f41ea0e46edae87b8e4dfa995f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:24.816890 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0538f41ea0e46edae87b8e4dfa995f0" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:24.816890 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0538f41ea0e46edae87b8e4dfa995f0" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:24.817008 31755 raft_consensus.cc:2454] T d0538f41ea0e46edae87b8e4dfa995f0 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:24.817013 31626 raft_consensus.cc:2454] T d0538f41ea0e46edae87b8e4dfa995f0 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:24.817252 31428 leader_election.cc:302] T d0538f41ea0e46edae87b8e4dfa995f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:24.817454 3829 raft_consensus.cc:2781] T d0538f41ea0e46edae87b8e4dfa995f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:24.817500 3829 raft_consensus.cc:482] T d0538f41ea0e46edae87b8e4dfa995f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:24.817530 3829 raft_consensus.cc:3037] T d0538f41ea0e46edae87b8e4dfa995f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:24.818167 3829 raft_consensus.cc:504] T d0538f41ea0e46edae87b8e4dfa995f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:24.818446 3829 leader_election.cc:288] T d0538f41ea0e46edae87b8e4dfa995f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:24.818750 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0538f41ea0e46edae87b8e4dfa995f0" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:24.818769 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0538f41ea0e46edae87b8e4dfa995f0" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:24.818862 31755 raft_consensus.cc:3037] T d0538f41ea0e46edae87b8e4dfa995f0 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:24.818871 31626 raft_consensus.cc:3037] T d0538f41ea0e46edae87b8e4dfa995f0 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:24.819713 31755 raft_consensus.cc:2454] T d0538f41ea0e46edae87b8e4dfa995f0 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:24.819713 31626 raft_consensus.cc:2454] T d0538f41ea0e46edae87b8e4dfa995f0 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:24.819979 31429 leader_election.cc:302] T d0538f41ea0e46edae87b8e4dfa995f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, 315915efcf2b413ca7f9d874338e1b2d; no voters:
I20240502 20:46:24.820160 3829 raft_consensus.cc:2781] T d0538f41ea0e46edae87b8e4dfa995f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:24.820243 3829 raft_consensus.cc:686] T d0538f41ea0e46edae87b8e4dfa995f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Becoming Leader. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Running, Role: LEADER
I20240502 20:46:24.820366 3829 consensus_queue.cc:237] T d0538f41ea0e46edae87b8e4dfa995f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:24.822091 29041 catalog_manager.cc:5484] T d0538f41ea0e46edae87b8e4dfa995f0 P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69). New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:24.841296 29295 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:24.842834 31465 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:24.842844 31605 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:24.854617 29163 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:24.859962 29557 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:24.862877 31735 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:24.872812 31867 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:24.891743 29041 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:24.897514 31605 tablet_service.cc:1467] Processing CreateTablet for tablet 035f0261307340fa88391a8c2fe564cd (DEFAULT_TABLE table=rebalancer_dynamic_table [id=5d82e87b345646069e03d8f5b37ec573]), partition=
I20240502 20:46:24.897758 31605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 035f0261307340fa88391a8c2fe564cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:24.900369 3773 tablet_bootstrap.cc:492] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:24.901269 3773 tablet_bootstrap.cc:654] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:24.902240 31465 tablet_service.cc:1467] Processing CreateTablet for tablet 035f0261307340fa88391a8c2fe564cd (DEFAULT_TABLE table=rebalancer_dynamic_table [id=5d82e87b345646069e03d8f5b37ec573]), partition=
I20240502 20:46:24.902479 31465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 035f0261307340fa88391a8c2fe564cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:24.902757 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 035f0261307340fa88391a8c2fe564cd (DEFAULT_TABLE table=rebalancer_dynamic_table [id=5d82e87b345646069e03d8f5b37ec573]), partition=
I20240502 20:46:24.902987 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 035f0261307340fa88391a8c2fe564cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:24.903126 29424 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:24.905557 3845 tablet_bootstrap.cc:492] T 035f0261307340fa88391a8c2fe564cd P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:24.906340 4104 tablet_bootstrap.cc:492] T 035f0261307340fa88391a8c2fe564cd P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:24.906399 3845 tablet_bootstrap.cc:654] T 035f0261307340fa88391a8c2fe564cd P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:24.907205 4104 tablet_bootstrap.cc:654] T 035f0261307340fa88391a8c2fe564cd P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:24.907405 3845 tablet_bootstrap.cc:492] T 035f0261307340fa88391a8c2fe564cd P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:24.907475 3845 ts_tablet_manager.cc:1367] T 035f0261307340fa88391a8c2fe564cd P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:24.907896 3845 raft_consensus.cc:348] T 035f0261307340fa88391a8c2fe564cd P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:24.908020 3845 raft_consensus.cc:374] T 035f0261307340fa88391a8c2fe564cd P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:24.908049 3845 raft_consensus.cc:729] T 035f0261307340fa88391a8c2fe564cd P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:24.908155 3845 consensus_queue.cc:260] T 035f0261307340fa88391a8c2fe564cd P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:24.908423 3845 ts_tablet_manager.cc:1398] T 035f0261307340fa88391a8c2fe564cd P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:24.909760 4104 tablet_bootstrap.cc:492] T 035f0261307340fa88391a8c2fe564cd P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:24.909843 4104 ts_tablet_manager.cc:1367] T 035f0261307340fa88391a8c2fe564cd P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:24.910421 4104 raft_consensus.cc:348] T 035f0261307340fa88391a8c2fe564cd P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:24.910542 4104 raft_consensus.cc:374] T 035f0261307340fa88391a8c2fe564cd P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:24.910570 4104 raft_consensus.cc:729] T 035f0261307340fa88391a8c2fe564cd P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:24.910689 4104 consensus_queue.cc:260] T 035f0261307340fa88391a8c2fe564cd P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:24.911001 4104 ts_tablet_manager.cc:1398] T 035f0261307340fa88391a8c2fe564cd P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:24.915544 3773 tablet_bootstrap.cc:492] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:24.915654 3773 ts_tablet_manager.cc:1367] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 20:46:24.916071 3773 raft_consensus.cc:348] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:24.916169 3773 raft_consensus.cc:374] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:24.916195 3773 raft_consensus.cc:729] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:24.916297 3773 consensus_queue.cc:260] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:24.916524 3773 ts_tablet_manager.cc:1398] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:24.952077 29577 consensus_queue.cc:237] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:24.953270 31888 raft_consensus.cc:1261] T 8c21a6762eaa4ba4ac1c91a42fb37849 P e6b60803d0764d228c863e2da65e3691 [term 2 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 20:46:24.953436 29444 raft_consensus.cc:1261] T 8c21a6762eaa4ba4ac1c91a42fb37849 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 2 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 20:46:24.953658 4216 consensus_queue.cc:1035] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 20:46:24.953859 3290 consensus_queue.cc:1035] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 20:46:24.955629 4219 raft_consensus.cc:2932] T 8c21a6762eaa4ba4ac1c91a42fb37849 P db17f26a73f3447b92aa9a947e3a7057 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:24.956039 29444 raft_consensus.cc:2932] T 8c21a6762eaa4ba4ac1c91a42fb37849 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:24.956072 31888 raft_consensus.cc:2932] T 8c21a6762eaa4ba4ac1c91a42fb37849 P e6b60803d0764d228c863e2da65e3691 [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:24.959766 29041 catalog_manager.cc:5484] T 8c21a6762eaa4ba4ac1c91a42fb37849 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index 4 to 6, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) evicted. New cstate: current_term: 2 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:24.962584 29163 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c21a6762eaa4ba4ac1c91a42fb37849 with delete_type TABLET_DATA_TOMBSTONED (TS 7796ea3fba294d17a34386873969431d not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:24.963418 4220 tablet_replica.cc:330] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:24.963666 4220 raft_consensus.cc:2229] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:24.963788 4220 raft_consensus.cc:2258] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:24.964434 4220 ts_tablet_manager.cc:1875] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:24.966607 4220 ts_tablet_manager.cc:1888] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 20:46:24.966724 4220 log.cc:1196] T 8c21a6762eaa4ba4ac1c91a42fb37849 P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/8c21a6762eaa4ba4ac1c91a42fb37849
I20240502 20:46:24.967283 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 8c21a6762eaa4ba4ac1c91a42fb37849 (table rebalance_test_table_19 [id=5fd0afffa41641b9b8de6545456ff39a]) successfully deleted
I20240502 20:46:25.002317 29444 consensus_queue.cc:237] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:25.003656 29577 raft_consensus.cc:1261] T 3c3bdc0b9e934ae2b81317a371817563 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 20:46:25.003865 29380 consensus_peers.cc:485] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3c3bdc0b9e934ae2b81317a371817563. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:25.004223 4226 consensus_queue.cc:1035] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:25.004703 29313 raft_consensus.cc:1261] T 3c3bdc0b9e934ae2b81317a371817563 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:25.005077 4226 consensus_queue.cc:1035] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:25.006745 3480 raft_consensus.cc:2932] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:25.008478 29577 raft_consensus.cc:2932] T 3c3bdc0b9e934ae2b81317a371817563 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:25.009220 29042 catalog_manager.cc:5484] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:25.010116 29313 raft_consensus.cc:2932] T 3c3bdc0b9e934ae2b81317a371817563 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:25.033052 31494 consensus_queue.cc:237] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:25.034354 31755 raft_consensus.cc:1261] T 63a41aabb95d4e8a9071c7673b257c47 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:25.034345 29313 raft_consensus.cc:1261] T 63a41aabb95d4e8a9071c7673b257c47 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 20:46:25.034751 31430 consensus_peers.cc:485] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): Couldn't send request to peer dbcdc6ac635b44b185c3eba63f66e9d6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 63a41aabb95d4e8a9071c7673b257c47. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:25.034770 3167 consensus_queue.cc:1035] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:25.034977 4278 consensus_queue.cc:1035] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:25.036739 4278 raft_consensus.cc:2932] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:25.037166 29313 raft_consensus.cc:2932] T 63a41aabb95d4e8a9071c7673b257c47 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:25.040447 29041 catalog_manager.cc:5484] T 63a41aabb95d4e8a9071c7673b257c47 P 09258ac095664fa99caf0707262cde13 reported cstate change: config changed from index -1 to 2, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) added. New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:25.041474 31755 raft_consensus.cc:2932] T 63a41aabb95d4e8a9071c7673b257c47 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:25.056551 29577 consensus_queue.cc:237] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:25.057823 31755 raft_consensus.cc:1261] T 28d6a4d9c92141f794d6ee04335cf9b8 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:25.058050 31494 raft_consensus.cc:1261] T 28d6a4d9c92141f794d6ee04335cf9b8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 20:46:25.058394 29511 consensus_peers.cc:485] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 -> Peer e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): Couldn't send request to peer e6b60803d0764d228c863e2da65e3691. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 28d6a4d9c92141f794d6ee04335cf9b8. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:25.058494 4216 consensus_queue.cc:1035] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:25.058692 4219 consensus_queue.cc:1035] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:25.060346 4219 raft_consensus.cc:2932] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:25.060747 31494 raft_consensus.cc:2932] T 28d6a4d9c92141f794d6ee04335cf9b8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:25.061933 31755 raft_consensus.cc:2932] T 28d6a4d9c92141f794d6ee04335cf9b8 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:25.063409 29035 catalog_manager.cc:5484] T 28d6a4d9c92141f794d6ee04335cf9b8 P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: config changed from index -1 to 2, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:25.080763 29577 consensus_queue.cc:237] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:25.082053 29313 raft_consensus.cc:1261] T 9acc12c9f0464e07965c05285fd334f0 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:25.082170 29444 raft_consensus.cc:1261] T 9acc12c9f0464e07965c05285fd334f0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 20:46:25.082253 29510 consensus_peers.cc:485] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 -> Peer e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): Couldn't send request to peer e7ab6b9053664ee5aa71c7978e26fe6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9acc12c9f0464e07965c05285fd334f0. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:25.082518 4219 consensus_queue.cc:1035] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:25.082695 3290 consensus_queue.cc:1035] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:25.084262 3290 raft_consensus.cc:2932] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:25.084676 29444 raft_consensus.cc:2932] T 9acc12c9f0464e07965c05285fd334f0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:25.086064 29313 raft_consensus.cc:2932] T 9acc12c9f0464e07965c05285fd334f0 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:25.087960 29038 catalog_manager.cc:5484] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:25.094133 3898 consensus_queue.cc:1035] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:25.097283 4329 consensus_queue.cc:1035] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:25.107120 29444 consensus_queue.cc:237] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:25.108493 29175 raft_consensus.cc:1261] T 21ee3d7c6f764d0483b335da40e61efc P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:25.108498 29313 raft_consensus.cc:1261] T 21ee3d7c6f764d0483b335da40e61efc P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:25.108839 4261 consensus_queue.cc:1035] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:25.108994 3480 consensus_queue.cc:1035] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 20:46:25.109388 29381 consensus_peers.cc:485] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): Couldn't send request to peer 2a9c1af8cf814030bde56b67ca5c2fbe. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 21ee3d7c6f764d0483b335da40e61efc. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:25.110844 3480 raft_consensus.cc:2932] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:25.111245 29313 raft_consensus.cc:2932] T 21ee3d7c6f764d0483b335da40e61efc P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:25.113636 29041 catalog_manager.cc:5484] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:25.113718 29175 raft_consensus.cc:2932] T 21ee3d7c6f764d0483b335da40e61efc P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:25.159997 2289 raft_consensus.cc:482] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:25.160104 2289 raft_consensus.cc:504] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:25.160570 2289 leader_election.cc:288] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:25.161306 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "035f0261307340fa88391a8c2fe564cd" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:25.161429 31494 raft_consensus.cc:2454] T 035f0261307340fa88391a8c2fe564cd P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:25.161645 29313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "035f0261307340fa88391a8c2fe564cd" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:25.161767 29313 raft_consensus.cc:2454] T 035f0261307340fa88391a8c2fe564cd P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:25.162331 31561 leader_election.cc:302] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 315915efcf2b413ca7f9d874338e1b2d; no voters:
I20240502 20:46:25.162636 2289 raft_consensus.cc:2781] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:25.162706 2289 raft_consensus.cc:482] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:25.162744 2289 raft_consensus.cc:3037] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:25.163713 2289 raft_consensus.cc:504] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:25.163995 2289 leader_election.cc:288] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Requested vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:25.164353 29313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "035f0261307340fa88391a8c2fe564cd" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:25.164378 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "035f0261307340fa88391a8c2fe564cd" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:25.164455 29313 raft_consensus.cc:3037] T 035f0261307340fa88391a8c2fe564cd P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:25.164474 31494 raft_consensus.cc:3037] T 035f0261307340fa88391a8c2fe564cd P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:25.165349 31494 raft_consensus.cc:2454] T 035f0261307340fa88391a8c2fe564cd P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:25.165365 29313 raft_consensus.cc:2454] T 035f0261307340fa88391a8c2fe564cd P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:25.165629 31561 leader_election.cc:302] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 315915efcf2b413ca7f9d874338e1b2d; no voters:
I20240502 20:46:25.165807 2289 raft_consensus.cc:2781] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:25.165896 2289 raft_consensus.cc:686] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Becoming Leader. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Running, Role: LEADER
I20240502 20:46:25.166025 2289 consensus_queue.cc:237] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:25.168138 29040 catalog_manager.cc:5484] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70). New cstate: current_term: 1 leader_uuid: "315915efcf2b413ca7f9d874338e1b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:25.217962 31465 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:25.221253 31605 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:25.221587 29557 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:25.222441 29295 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:25.224653 29424 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:25.240703 31867 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:25.242254 29163 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:25.242730 31735 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:25.309047 29037 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:25.313550 29313 raft_consensus.cc:1261] T 035f0261307340fa88391a8c2fe564cd P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 315915efcf2b413ca7f9d874338e1b2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:25.313738 31494 raft_consensus.cc:1261] T 035f0261307340fa88391a8c2fe564cd P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Refusing update from remote peer 315915efcf2b413ca7f9d874338e1b2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:25.314241 2289 consensus_queue.cc:1035] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:25.314558 2289 consensus_queue.cc:1035] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:25.317575 29041 catalog_manager.cc:5576] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887) does not have the latest schema for tablet 035f0261307340fa88391a8c2fe564cd (table rebalancer_dynamic_table [id=5d82e87b345646069e03d8f5b37ec573]). Expected version 1 got 0
I20240502 20:46:25.318425 4438 tablet.cc:1722] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:25.318766 4438 tablet.cc:1620] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:25.319514 29041 catalog_manager.cc:5576] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945) does not have the latest schema for tablet 035f0261307340fa88391a8c2fe564cd (table rebalancer_dynamic_table [id=5d82e87b345646069e03d8f5b37ec573]). Expected version 1 got 0
I20240502 20:46:25.322351 4439 tablet.cc:1722] T 035f0261307340fa88391a8c2fe564cd P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:25.322993 29037 catalog_manager.cc:5576] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945) does not have the latest schema for tablet 035f0261307340fa88391a8c2fe564cd (table rebalancer_dynamic_table [id=5d82e87b345646069e03d8f5b37ec573]). Expected version 1 got 0
I20240502 20:46:25.323755 4439 tablet.cc:1620] T 035f0261307340fa88391a8c2fe564cd P 09258ac095664fa99caf0707262cde13: MemRowSet was empty: no flush needed.
I20240502 20:46:25.327551 29037 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=5d82e87b345646069e03d8f5b37ec573] alter complete (version 1)
I20240502 20:46:25.330397 4440 tablet.cc:1722] T 035f0261307340fa88391a8c2fe564cd P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:25.330776 4440 tablet.cc:1620] T 035f0261307340fa88391a8c2fe564cd P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:25.349396 4281 consensus_queue.cc:1035] T d0538f41ea0e46edae87b8e4dfa995f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:25.353058 4019 consensus_queue.cc:1035] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:25.355221 3829 consensus_queue.cc:1035] T d0538f41ea0e46edae87b8e4dfa995f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:25.361018 29034 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_34"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:25.365538 4019 consensus_queue.cc:1035] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:25.370743 31605 tablet_service.cc:1467] Processing CreateTablet for tablet b699c00ff92b41e9ae24e5982ced3ef9 (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=c292b10269a84c8e83a750d69bb6f95b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:25.371039 31605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b699c00ff92b41e9ae24e5982ced3ef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:25.371572 31735 tablet_service.cc:1467] Processing CreateTablet for tablet b7c80c3b3a5d4505b7dbbac6bab1de59 (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=c292b10269a84c8e83a750d69bb6f95b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:25.371798 31735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7c80c3b3a5d4505b7dbbac6bab1de59. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:25.373739 29163 tablet_service.cc:1467] Processing CreateTablet for tablet 2a00833b5b404e63b0f720b5c8fb528d (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=c292b10269a84c8e83a750d69bb6f95b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:25.374030 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a00833b5b404e63b0f720b5c8fb528d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:25.374222 31465 tablet_service.cc:1467] Processing CreateTablet for tablet b699c00ff92b41e9ae24e5982ced3ef9 (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=c292b10269a84c8e83a750d69bb6f95b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:25.374502 31465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b699c00ff92b41e9ae24e5982ced3ef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:25.376569 3773 tablet_bootstrap.cc:492] T b699c00ff92b41e9ae24e5982ced3ef9 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:25.376713 31474 tablet_service.cc:1467] Processing CreateTablet for tablet b7c80c3b3a5d4505b7dbbac6bab1de59 (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=c292b10269a84c8e83a750d69bb6f95b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:25.376935 31474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7c80c3b3a5d4505b7dbbac6bab1de59. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:25.377537 3773 tablet_bootstrap.cc:654] T b699c00ff92b41e9ae24e5982ced3ef9 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:25.378115 31867 tablet_service.cc:1467] Processing CreateTablet for tablet b699c00ff92b41e9ae24e5982ced3ef9 (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=c292b10269a84c8e83a750d69bb6f95b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:25.378379 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b699c00ff92b41e9ae24e5982ced3ef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:25.379009 3845 tablet_bootstrap.cc:492] T b699c00ff92b41e9ae24e5982ced3ef9 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:25.379917 3845 tablet_bootstrap.cc:654] T b699c00ff92b41e9ae24e5982ced3ef9 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:25.380620 29424 tablet_service.cc:1467] Processing CreateTablet for tablet b7c80c3b3a5d4505b7dbbac6bab1de59 (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=c292b10269a84c8e83a750d69bb6f95b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:25.380692 4450 tablet_bootstrap.cc:492] T b699c00ff92b41e9ae24e5982ced3ef9 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:25.380892 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7c80c3b3a5d4505b7dbbac6bab1de59. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:25.381633 4450 tablet_bootstrap.cc:654] T b699c00ff92b41e9ae24e5982ced3ef9 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:25.382333 31605 tablet_service.cc:1467] Processing CreateTablet for tablet 2a00833b5b404e63b0f720b5c8fb528d (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=c292b10269a84c8e83a750d69bb6f95b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:25.382548 4446 tablet_bootstrap.cc:492] T b7c80c3b3a5d4505b7dbbac6bab1de59 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:25.382616 31605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a00833b5b404e63b0f720b5c8fb528d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:25.382972 4447 tablet_bootstrap.cc:492] T 2a00833b5b404e63b0f720b5c8fb528d P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:25.383493 4446 tablet_bootstrap.cc:654] T b7c80c3b3a5d4505b7dbbac6bab1de59 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:25.374222 31475 tablet_service.cc:1467] Processing CreateTablet for tablet 2a00833b5b404e63b0f720b5c8fb528d (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=c292b10269a84c8e83a750d69bb6f95b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:25.383812 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a00833b5b404e63b0f720b5c8fb528d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:25.383850 4447 tablet_bootstrap.cc:654] T 2a00833b5b404e63b0f720b5c8fb528d P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:25.385203 3845 tablet_bootstrap.cc:492] T b699c00ff92b41e9ae24e5982ced3ef9 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:25.385284 3845 ts_tablet_manager.cc:1367] T b699c00ff92b41e9ae24e5982ced3ef9 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20240502 20:46:25.385638 4450 tablet_bootstrap.cc:492] T b699c00ff92b41e9ae24e5982ced3ef9 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:25.385720 4450 ts_tablet_manager.cc:1367] T b699c00ff92b41e9ae24e5982ced3ef9 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 20:46:25.385711 3845 raft_consensus.cc:348] T b699c00ff92b41e9ae24e5982ced3ef9 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:25.385818 3845 raft_consensus.cc:374] T b699c00ff92b41e9ae24e5982ced3ef9 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:25.385844 3845 raft_consensus.cc:729] T b699c00ff92b41e9ae24e5982ced3ef9 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:25.385942 3845 consensus_queue.cc:260] T b699c00ff92b41e9ae24e5982ced3ef9 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:25.386188 3845 ts_tablet_manager.cc:1398] T b699c00ff92b41e9ae24e5982ced3ef9 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:25.386179 4450 raft_consensus.cc:348] T b699c00ff92b41e9ae24e5982ced3ef9 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:25.386287 4450 raft_consensus.cc:374] T b699c00ff92b41e9ae24e5982ced3ef9 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:25.386322 4450 raft_consensus.cc:729] T b699c00ff92b41e9ae24e5982ced3ef9 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:25.386351 3845 tablet_bootstrap.cc:492] T b7c80c3b3a5d4505b7dbbac6bab1de59 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:25.386444 4450 consensus_queue.cc:260] T b699c00ff92b41e9ae24e5982ced3ef9 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:25.386781 4450 ts_tablet_manager.cc:1398] T b699c00ff92b41e9ae24e5982ced3ef9 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:25.387176 3845 tablet_bootstrap.cc:654] T b7c80c3b3a5d4505b7dbbac6bab1de59 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:25.391304 3845 tablet_bootstrap.cc:492] T b7c80c3b3a5d4505b7dbbac6bab1de59 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:25.391424 3845 ts_tablet_manager.cc:1367] T b7c80c3b3a5d4505b7dbbac6bab1de59 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 20:46:25.391837 3845 raft_consensus.cc:348] T b7c80c3b3a5d4505b7dbbac6bab1de59 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:25.391976 3845 raft_consensus.cc:374] T b7c80c3b3a5d4505b7dbbac6bab1de59 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:25.392052 3845 raft_consensus.cc:729] T b7c80c3b3a5d4505b7dbbac6bab1de59 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:25.392217 3845 consensus_queue.cc:260] T b7c80c3b3a5d4505b7dbbac6bab1de59 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:25.392877 4451 tablet_bootstrap.cc:492] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:25.393710 4451 tablet_bootstrap.cc:654] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:25.394033 3845 ts_tablet_manager.cc:1398] T b7c80c3b3a5d4505b7dbbac6bab1de59 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20240502 20:46:25.394222 3845 tablet_bootstrap.cc:492] T 2a00833b5b404e63b0f720b5c8fb528d P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:25.395143 4447 tablet_bootstrap.cc:492] T 2a00833b5b404e63b0f720b5c8fb528d P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:25.395241 4447 ts_tablet_manager.cc:1367] T 2a00833b5b404e63b0f720b5c8fb528d P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 20:46:25.396082 3845 tablet_bootstrap.cc:654] T 2a00833b5b404e63b0f720b5c8fb528d P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:25.395665 4447 raft_consensus.cc:348] T 2a00833b5b404e63b0f720b5c8fb528d P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:25.397729 4447 raft_consensus.cc:374] T 2a00833b5b404e63b0f720b5c8fb528d P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:25.397759 4447 raft_consensus.cc:729] T 2a00833b5b404e63b0f720b5c8fb528d P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:25.397866 4447 consensus_queue.cc:260] T 2a00833b5b404e63b0f720b5c8fb528d P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:25.398902 4447 ts_tablet_manager.cc:1398] T 2a00833b5b404e63b0f720b5c8fb528d P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:25.401961 3773 tablet_bootstrap.cc:492] T b699c00ff92b41e9ae24e5982ced3ef9 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:25.402055 3773 ts_tablet_manager.cc:1367] T b699c00ff92b41e9ae24e5982ced3ef9 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.000s
I20240502 20:46:25.402119 4451 tablet_bootstrap.cc:492] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:25.402195 4451 ts_tablet_manager.cc:1367] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 20:46:25.402527 3773 raft_consensus.cc:348] T b699c00ff92b41e9ae24e5982ced3ef9 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:25.402627 3773 raft_consensus.cc:374] T b699c00ff92b41e9ae24e5982ced3ef9 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:25.402676 3773 raft_consensus.cc:729] T b699c00ff92b41e9ae24e5982ced3ef9 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:25.402741 4451 raft_consensus.cc:348] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:25.402787 3773 consensus_queue.cc:260] T b699c00ff92b41e9ae24e5982ced3ef9 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:25.402827 4451 raft_consensus.cc:374] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:25.402855 4451 raft_consensus.cc:729] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:25.403045 3773 ts_tablet_manager.cc:1398] T b699c00ff92b41e9ae24e5982ced3ef9 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:25.403216 3773 tablet_bootstrap.cc:492] T 2a00833b5b404e63b0f720b5c8fb528d P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:25.404098 3773 tablet_bootstrap.cc:654] T 2a00833b5b404e63b0f720b5c8fb528d P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:25.405019 4451 consensus_queue.cc:260] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:25.408030 29295 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:25.413415 3773 tablet_bootstrap.cc:492] T 2a00833b5b404e63b0f720b5c8fb528d P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:25.413545 3773 ts_tablet_manager.cc:1367] T 2a00833b5b404e63b0f720b5c8fb528d P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 20:46:25.413739 4470 ts_tablet_manager.cc:902] T 3c3bdc0b9e934ae2b81317a371817563 P 315915efcf2b413ca7f9d874338e1b2d: Initiating tablet copy from peer dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:25.414081 3773 raft_consensus.cc:348] T 2a00833b5b404e63b0f720b5c8fb528d P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:25.414178 3773 raft_consensus.cc:374] T 2a00833b5b404e63b0f720b5c8fb528d P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:25.414211 3773 raft_consensus.cc:729] T 2a00833b5b404e63b0f720b5c8fb528d P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:25.414315 3773 consensus_queue.cc:260] T 2a00833b5b404e63b0f720b5c8fb528d P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:25.414604 3773 ts_tablet_manager.cc:1398] T 2a00833b5b404e63b0f720b5c8fb528d P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:25.416826 29424 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:25.419098 4470 tablet_copy_client.cc:308] T 3c3bdc0b9e934ae2b81317a371817563 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.67:44095
I20240502 20:46:25.419618 29465 tablet_copy_service.cc:140] P dbcdc6ac635b44b185c3eba63f66e9d6: Received BeginTabletCopySession request for tablet 3c3bdc0b9e934ae2b81317a371817563 from peer 315915efcf2b413ca7f9d874338e1b2d ({username='slave'} at 127.22.45.70:49775)
I20240502 20:46:25.419711 29465 tablet_copy_service.cc:161] P dbcdc6ac635b44b185c3eba63f66e9d6: Beginning new tablet copy session on tablet 3c3bdc0b9e934ae2b81317a371817563 from peer 315915efcf2b413ca7f9d874338e1b2d at {username='slave'} at 127.22.45.70:49775: session id = 315915efcf2b413ca7f9d874338e1b2d-3c3bdc0b9e934ae2b81317a371817563
I20240502 20:46:25.420692 29465 tablet_copy_source_session.cc:205] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:25.423434 3845 tablet_bootstrap.cc:492] T 2a00833b5b404e63b0f720b5c8fb528d P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:25.423542 3845 ts_tablet_manager.cc:1367] T 2a00833b5b404e63b0f720b5c8fb528d P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.029s user 0.001s sys 0.001s
I20240502 20:46:25.423990 3845 raft_consensus.cc:348] T 2a00833b5b404e63b0f720b5c8fb528d P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:25.424103 3845 raft_consensus.cc:374] T 2a00833b5b404e63b0f720b5c8fb528d P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:25.424141 3845 raft_consensus.cc:729] T 2a00833b5b404e63b0f720b5c8fb528d P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:25.424247 3845 consensus_queue.cc:260] T 2a00833b5b404e63b0f720b5c8fb528d P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:25.424819 3845 ts_tablet_manager.cc:1398] T 2a00833b5b404e63b0f720b5c8fb528d P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 20:46:25.428367 4451 ts_tablet_manager.cc:1398] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.026s user 0.001s sys 0.000s
I20240502 20:46:25.437836 31735 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:25.438838 31475 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:25.442256 29041 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:25.446607 29163 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:25.446864 29557 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:25.447458 4446 tablet_bootstrap.cc:492] T b7c80c3b3a5d4505b7dbbac6bab1de59 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:25.447552 4446 ts_tablet_manager.cc:1367] T b7c80c3b3a5d4505b7dbbac6bab1de59 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.065s user 0.002s sys 0.000s
I20240502 20:46:25.448346 4446 raft_consensus.cc:348] T b7c80c3b3a5d4505b7dbbac6bab1de59 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:25.448524 4446 raft_consensus.cc:374] T b7c80c3b3a5d4505b7dbbac6bab1de59 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:25.448599 4446 raft_consensus.cc:729] T b7c80c3b3a5d4505b7dbbac6bab1de59 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:25.448777 4446 consensus_queue.cc:260] T b7c80c3b3a5d4505b7dbbac6bab1de59 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:25.449905 4446 ts_tablet_manager.cc:1398] T b7c80c3b3a5d4505b7dbbac6bab1de59 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:25.450359 4470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c3bdc0b9e934ae2b81317a371817563. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:25.452848 4470 tablet_copy_client.cc:783] T 3c3bdc0b9e934ae2b81317a371817563 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:25.453269 31605 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:25.453620 4470 tablet_copy_client.cc:647] T 3c3bdc0b9e934ae2b81317a371817563 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:25.453696 31867 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:25.454725 4470 tablet_copy_client.cc:523] T 3c3bdc0b9e934ae2b81317a371817563 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:25.455907 4470 tablet_bootstrap.cc:492] T 3c3bdc0b9e934ae2b81317a371817563 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:25.469719 4438 tablet.cc:1722] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:25.469986 4438 tablet.cc:1620] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:25.470137 3829 raft_consensus.cc:482] T b699c00ff92b41e9ae24e5982ced3ef9 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:25.470219 3829 raft_consensus.cc:504] T b699c00ff92b41e9ae24e5982ced3ef9 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:25.470598 3829 leader_election.cc:288] T b699c00ff92b41e9ae24e5982ced3ef9 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:25.471035 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b699c00ff92b41e9ae24e5982ced3ef9" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:25.471161 31888 raft_consensus.cc:2454] T b699c00ff92b41e9ae24e5982ced3ef9 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:25.471460 31429 leader_election.cc:302] T b699c00ff92b41e9ae24e5982ced3ef9 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:25.472967 3829 raft_consensus.cc:2781] T b699c00ff92b41e9ae24e5982ced3ef9 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:25.473024 3829 raft_consensus.cc:482] T b699c00ff92b41e9ae24e5982ced3ef9 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:25.473062 3829 raft_consensus.cc:3037] T b699c00ff92b41e9ae24e5982ced3ef9 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:25.473685 4439 tablet.cc:1722] T 035f0261307340fa88391a8c2fe564cd P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:25.473981 4439 tablet.cc:1620] T 035f0261307340fa88391a8c2fe564cd P 09258ac095664fa99caf0707262cde13: MemRowSet was empty: no flush needed.
I20240502 20:46:25.473961 3829 raft_consensus.cc:504] T b699c00ff92b41e9ae24e5982ced3ef9 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:25.474195 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b699c00ff92b41e9ae24e5982ced3ef9" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:25.474298 31626 raft_consensus.cc:2454] T b699c00ff92b41e9ae24e5982ced3ef9 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:25.474303 3829 leader_election.cc:288] T b699c00ff92b41e9ae24e5982ced3ef9 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:25.474718 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b699c00ff92b41e9ae24e5982ced3ef9" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:25.474807 31626 raft_consensus.cc:3037] T b699c00ff92b41e9ae24e5982ced3ef9 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:25.474882 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b699c00ff92b41e9ae24e5982ced3ef9" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:25.474979 31888 raft_consensus.cc:3037] T b699c00ff92b41e9ae24e5982ced3ef9 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:25.475576 31626 raft_consensus.cc:2454] T b699c00ff92b41e9ae24e5982ced3ef9 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:25.475896 31429 leader_election.cc:302] T b699c00ff92b41e9ae24e5982ced3ef9 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, 315915efcf2b413ca7f9d874338e1b2d; no voters:
I20240502 20:46:25.475909 31888 raft_consensus.cc:2454] T b699c00ff92b41e9ae24e5982ced3ef9 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:25.478610 3829 raft_consensus.cc:2781] T b699c00ff92b41e9ae24e5982ced3ef9 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:25.478688 3829 raft_consensus.cc:686] T b699c00ff92b41e9ae24e5982ced3ef9 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Becoming Leader. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Running, Role: LEADER
I20240502 20:46:25.478762 4440 tablet.cc:1722] T 035f0261307340fa88391a8c2fe564cd P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:25.478793 3829 consensus_queue.cc:237] T b699c00ff92b41e9ae24e5982ced3ef9 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:25.479050 4440 tablet.cc:1620] T 035f0261307340fa88391a8c2fe564cd P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:25.480743 29034 catalog_manager.cc:5484] T b699c00ff92b41e9ae24e5982ced3ef9 P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69). New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:25.483676 29041 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=5d82e87b345646069e03d8f5b37ec573] alter complete (version 2)
I20240502 20:46:25.504834 3167 raft_consensus.cc:482] T 2a00833b5b404e63b0f720b5c8fb528d P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:25.504927 3167 raft_consensus.cc:504] T 2a00833b5b404e63b0f720b5c8fb528d P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:25.505295 3167 leader_election.cc:288] T 2a00833b5b404e63b0f720b5c8fb528d P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:25.505668 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a00833b5b404e63b0f720b5c8fb528d" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:25.505784 31626 raft_consensus.cc:2454] T 2a00833b5b404e63b0f720b5c8fb528d P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:25.506142 31429 leader_election.cc:302] T 2a00833b5b404e63b0f720b5c8fb528d P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, 315915efcf2b413ca7f9d874338e1b2d; no voters:
I20240502 20:46:25.506283 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a00833b5b404e63b0f720b5c8fb528d" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:25.506405 29175 raft_consensus.cc:2454] T 2a00833b5b404e63b0f720b5c8fb528d P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:25.507352 3167 raft_consensus.cc:2781] T 2a00833b5b404e63b0f720b5c8fb528d P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:25.507413 3167 raft_consensus.cc:482] T 2a00833b5b404e63b0f720b5c8fb528d P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:25.507449 3167 raft_consensus.cc:3037] T 2a00833b5b404e63b0f720b5c8fb528d P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:25.508330 3167 raft_consensus.cc:504] T 2a00833b5b404e63b0f720b5c8fb528d P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:25.508956 3167 leader_election.cc:288] T 2a00833b5b404e63b0f720b5c8fb528d P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:25.509356 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a00833b5b404e63b0f720b5c8fb528d" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:25.509460 31626 raft_consensus.cc:3037] T 2a00833b5b404e63b0f720b5c8fb528d P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:25.509478 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a00833b5b404e63b0f720b5c8fb528d" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:25.509570 29175 raft_consensus.cc:3037] T 2a00833b5b404e63b0f720b5c8fb528d P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:25.510336 31626 raft_consensus.cc:2454] T 2a00833b5b404e63b0f720b5c8fb528d P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:25.510411 29175 raft_consensus.cc:2454] T 2a00833b5b404e63b0f720b5c8fb528d P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:25.510664 31429 leader_election.cc:302] T 2a00833b5b404e63b0f720b5c8fb528d P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, 315915efcf2b413ca7f9d874338e1b2d; no voters:
I20240502 20:46:25.511133 3167 raft_consensus.cc:2781] T 2a00833b5b404e63b0f720b5c8fb528d P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:25.511209 3167 raft_consensus.cc:686] T 2a00833b5b404e63b0f720b5c8fb528d P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Becoming Leader. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Running, Role: LEADER
I20240502 20:46:25.511332 3167 consensus_queue.cc:237] T 2a00833b5b404e63b0f720b5c8fb528d P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:25.512506 4470 tablet_bootstrap.cc:492] T 3c3bdc0b9e934ae2b81317a371817563 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:25.512797 4470 tablet_bootstrap.cc:492] T 3c3bdc0b9e934ae2b81317a371817563 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap complete.
I20240502 20:46:25.512965 4470 ts_tablet_manager.cc:1367] T 3c3bdc0b9e934ae2b81317a371817563 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.057s user 0.024s sys 0.000s
I20240502 20:46:25.513418 29034 catalog_manager.cc:5484] T 2a00833b5b404e63b0f720b5c8fb528d P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69). New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:25.513631 4470 raft_consensus.cc:348] T 3c3bdc0b9e934ae2b81317a371817563 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:25.513738 4470 raft_consensus.cc:729] T 3c3bdc0b9e934ae2b81317a371817563 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:25.513914 4470 consensus_queue.cc:260] T 3c3bdc0b9e934ae2b81317a371817563 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:25.514349 4470 ts_tablet_manager.cc:1398] T 3c3bdc0b9e934ae2b81317a371817563 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:25.514936 29465 tablet_copy_service.cc:342] P dbcdc6ac635b44b185c3eba63f66e9d6: Request end of tablet copy session 315915efcf2b413ca7f9d874338e1b2d-3c3bdc0b9e934ae2b81317a371817563 received from {username='slave'} at 127.22.45.70:49775
I20240502 20:46:25.515014 29465 tablet_copy_service.cc:433] P dbcdc6ac635b44b185c3eba63f66e9d6: ending tablet copy session 315915efcf2b413ca7f9d874338e1b2d-3c3bdc0b9e934ae2b81317a371817563 on tablet 3c3bdc0b9e934ae2b81317a371817563 with peer 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:25.526052 4341 raft_consensus.cc:482] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:25.526144 4341 raft_consensus.cc:504] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:25.526541 4341 leader_election.cc:288] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:25.526837 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7c80c3b3a5d4505b7dbbac6bab1de59" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:25.526907 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7c80c3b3a5d4505b7dbbac6bab1de59" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:25.526978 31755 raft_consensus.cc:2454] T b7c80c3b3a5d4505b7dbbac6bab1de59 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:25.527019 31494 raft_consensus.cc:2454] T b7c80c3b3a5d4505b7dbbac6bab1de59 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:25.527276 29379 leader_election.cc:302] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dbcdc6ac635b44b185c3eba63f66e9d6, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:25.527482 4341 raft_consensus.cc:2781] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:25.527549 4341 raft_consensus.cc:482] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:25.527582 4341 raft_consensus.cc:3037] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:25.528527 4341 raft_consensus.cc:504] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:25.528847 4341 leader_election.cc:288] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:25.529121 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7c80c3b3a5d4505b7dbbac6bab1de59" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:25.529207 31755 raft_consensus.cc:3037] T b7c80c3b3a5d4505b7dbbac6bab1de59 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:25.529700 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7c80c3b3a5d4505b7dbbac6bab1de59" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:25.529790 31494 raft_consensus.cc:3037] T b7c80c3b3a5d4505b7dbbac6bab1de59 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:25.530100 31755 raft_consensus.cc:2454] T b7c80c3b3a5d4505b7dbbac6bab1de59 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:25.530547 29379 leader_election.cc:302] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dbcdc6ac635b44b185c3eba63f66e9d6, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:25.530676 31494 raft_consensus.cc:2454] T b7c80c3b3a5d4505b7dbbac6bab1de59 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:25.531002 4261 raft_consensus.cc:2781] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:25.531085 4261 raft_consensus.cc:686] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:25.531190 4261 consensus_queue.cc:237] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:25.533352 29034 catalog_manager.cc:5484] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:25.578049 4522 ts_tablet_manager.cc:902] T 63a41aabb95d4e8a9071c7673b257c47 P dbcdc6ac635b44b185c3eba63f66e9d6: Initiating tablet copy from peer 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:25.578800 4522 tablet_copy_client.cc:308] T 63a41aabb95d4e8a9071c7673b257c47 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.69:43887
I20240502 20:46:25.579512 31515 tablet_copy_service.cc:140] P 2a9c1af8cf814030bde56b67ca5c2fbe: Received BeginTabletCopySession request for tablet 63a41aabb95d4e8a9071c7673b257c47 from peer dbcdc6ac635b44b185c3eba63f66e9d6 ({username='slave'} at 127.22.45.67:46915)
I20240502 20:46:25.579656 31515 tablet_copy_service.cc:161] P 2a9c1af8cf814030bde56b67ca5c2fbe: Beginning new tablet copy session on tablet 63a41aabb95d4e8a9071c7673b257c47 from peer dbcdc6ac635b44b185c3eba63f66e9d6 at {username='slave'} at 127.22.45.67:46915: session id = dbcdc6ac635b44b185c3eba63f66e9d6-63a41aabb95d4e8a9071c7673b257c47
I20240502 20:46:25.581147 31515 tablet_copy_source_session.cc:205] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:25.584180 4522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63a41aabb95d4e8a9071c7673b257c47. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:25.586691 4522 tablet_copy_client.cc:783] T 63a41aabb95d4e8a9071c7673b257c47 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:25.586941 4522 tablet_copy_client.cc:647] T 63a41aabb95d4e8a9071c7673b257c47 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:25.589162 29040 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:25.590718 4522 tablet_copy_client.cc:523] T 63a41aabb95d4e8a9071c7673b257c47 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:25.592041 4522 tablet_bootstrap.cc:492] T 63a41aabb95d4e8a9071c7673b257c47 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:25.596873 4438 tablet.cc:1722] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:25.601617 4440 tablet.cc:1722] T 035f0261307340fa88391a8c2fe564cd P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:25.602994 29040 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=5d82e87b345646069e03d8f5b37ec573] alter complete (version 3)
I20240502 20:46:25.605057 4439 tablet.cc:1722] T 035f0261307340fa88391a8c2fe564cd P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:25.612207 4536 ts_tablet_manager.cc:902] T 28d6a4d9c92141f794d6ee04335cf9b8 P e6b60803d0764d228c863e2da65e3691: Initiating tablet copy from peer db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:25.614158 4536 tablet_copy_client.cc:308] T 28d6a4d9c92141f794d6ee04335cf9b8 P e6b60803d0764d228c863e2da65e3691: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.68:45897
I20240502 20:46:25.614619 29597 tablet_copy_service.cc:140] P db17f26a73f3447b92aa9a947e3a7057: Received BeginTabletCopySession request for tablet 28d6a4d9c92141f794d6ee04335cf9b8 from peer e6b60803d0764d228c863e2da65e3691 ({username='slave'} at 127.22.45.72:43997)
I20240502 20:46:25.614713 29597 tablet_copy_service.cc:161] P db17f26a73f3447b92aa9a947e3a7057: Beginning new tablet copy session on tablet 28d6a4d9c92141f794d6ee04335cf9b8 from peer e6b60803d0764d228c863e2da65e3691 at {username='slave'} at 127.22.45.72:43997: session id = e6b60803d0764d228c863e2da65e3691-28d6a4d9c92141f794d6ee04335cf9b8
I20240502 20:46:25.615614 29597 tablet_copy_source_session.cc:205] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:25.616746 4536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28d6a4d9c92141f794d6ee04335cf9b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:25.622123 31605 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:25.625897 29295 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:25.629377 31475 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:25.632441 4536 tablet_copy_client.cc:783] T 28d6a4d9c92141f794d6ee04335cf9b8 P e6b60803d0764d228c863e2da65e3691: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:25.632462 29557 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:25.634575 4522 tablet_bootstrap.cc:492] T 63a41aabb95d4e8a9071c7673b257c47 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:25.634809 4522 tablet_bootstrap.cc:492] T 63a41aabb95d4e8a9071c7673b257c47 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap complete.
I20240502 20:46:25.634946 4522 ts_tablet_manager.cc:1367] T 63a41aabb95d4e8a9071c7673b257c47 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.043s user 0.003s sys 0.002s
I20240502 20:46:25.635423 4522 raft_consensus.cc:348] T 63a41aabb95d4e8a9071c7673b257c47 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:25.635527 4522 raft_consensus.cc:729] T 63a41aabb95d4e8a9071c7673b257c47 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:25.635634 4522 consensus_queue.cc:260] T 63a41aabb95d4e8a9071c7673b257c47 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:25.635747 29163 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:25.635893 4522 ts_tablet_manager.cc:1398] T 63a41aabb95d4e8a9071c7673b257c47 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:25.637315 31515 tablet_copy_service.cc:342] P 2a9c1af8cf814030bde56b67ca5c2fbe: Request end of tablet copy session dbcdc6ac635b44b185c3eba63f66e9d6-63a41aabb95d4e8a9071c7673b257c47 received from {username='slave'} at 127.22.45.67:46915
I20240502 20:46:25.637398 31515 tablet_copy_service.cc:433] P 2a9c1af8cf814030bde56b67ca5c2fbe: ending tablet copy session dbcdc6ac635b44b185c3eba63f66e9d6-63a41aabb95d4e8a9071c7673b257c47 on tablet 63a41aabb95d4e8a9071c7673b257c47 with peer dbcdc6ac635b44b185c3eba63f66e9d6
I20240502 20:46:25.638049 4536 tablet_copy_client.cc:647] T 28d6a4d9c92141f794d6ee04335cf9b8 P e6b60803d0764d228c863e2da65e3691: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:25.639070 29424 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:25.639086 4536 tablet_copy_client.cc:523] T 28d6a4d9c92141f794d6ee04335cf9b8 P e6b60803d0764d228c863e2da65e3691: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:25.640296 4536 tablet_bootstrap.cc:492] T 28d6a4d9c92141f794d6ee04335cf9b8 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:25.647810 4536 tablet_bootstrap.cc:492] T 28d6a4d9c92141f794d6ee04335cf9b8 P e6b60803d0764d228c863e2da65e3691: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:25.648090 4536 tablet_bootstrap.cc:492] T 28d6a4d9c92141f794d6ee04335cf9b8 P e6b60803d0764d228c863e2da65e3691: Bootstrap complete.
I20240502 20:46:25.648260 4536 ts_tablet_manager.cc:1367] T 28d6a4d9c92141f794d6ee04335cf9b8 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.003s
I20240502 20:46:25.648676 4536 raft_consensus.cc:348] T 28d6a4d9c92141f794d6ee04335cf9b8 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:25.648773 4536 raft_consensus.cc:729] T 28d6a4d9c92141f794d6ee04335cf9b8 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:25.648903 4536 consensus_queue.cc:260] T 28d6a4d9c92141f794d6ee04335cf9b8 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:25.649178 4536 ts_tablet_manager.cc:1398] T 28d6a4d9c92141f794d6ee04335cf9b8 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:25.649597 29597 tablet_copy_service.cc:342] P db17f26a73f3447b92aa9a947e3a7057: Request end of tablet copy session e6b60803d0764d228c863e2da65e3691-28d6a4d9c92141f794d6ee04335cf9b8 received from {username='slave'} at 127.22.45.72:43997
I20240502 20:46:25.649660 29597 tablet_copy_service.cc:433] P db17f26a73f3447b92aa9a947e3a7057: ending tablet copy session e6b60803d0764d228c863e2da65e3691-28d6a4d9c92141f794d6ee04335cf9b8 on tablet 28d6a4d9c92141f794d6ee04335cf9b8 with peer e6b60803d0764d228c863e2da65e3691
I20240502 20:46:25.658869 31867 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:25.667129 4545 ts_tablet_manager.cc:902] T 9acc12c9f0464e07965c05285fd334f0 P e7ab6b9053664ee5aa71c7978e26fe6a: Initiating tablet copy from peer db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:25.674274 4545 tablet_copy_client.cc:308] T 9acc12c9f0464e07965c05285fd334f0 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.68:45897
I20240502 20:46:25.674687 29597 tablet_copy_service.cc:140] P db17f26a73f3447b92aa9a947e3a7057: Received BeginTabletCopySession request for tablet 9acc12c9f0464e07965c05285fd334f0 from peer e7ab6b9053664ee5aa71c7978e26fe6a ({username='slave'} at 127.22.45.71:51575)
I20240502 20:46:25.674779 29597 tablet_copy_service.cc:161] P db17f26a73f3447b92aa9a947e3a7057: Beginning new tablet copy session on tablet 9acc12c9f0464e07965c05285fd334f0 from peer e7ab6b9053664ee5aa71c7978e26fe6a at {username='slave'} at 127.22.45.71:51575: session id = e7ab6b9053664ee5aa71c7978e26fe6a-9acc12c9f0464e07965c05285fd334f0
I20240502 20:46:25.675706 29597 tablet_copy_source_session.cc:205] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:25.677443 4545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9acc12c9f0464e07965c05285fd334f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:25.677534 31735 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:25.681092 4545 tablet_copy_client.cc:783] T 9acc12c9f0464e07965c05285fd334f0 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:25.681246 4545 tablet_copy_client.cc:647] T 9acc12c9f0464e07965c05285fd334f0 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:25.682171 4545 tablet_copy_client.cc:523] T 9acc12c9f0464e07965c05285fd334f0 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:25.683142 4545 tablet_bootstrap.cc:492] T 9acc12c9f0464e07965c05285fd334f0 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:25.705343 29040 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:25.705492 29040 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:25.706756 4545 tablet_bootstrap.cc:492] T 9acc12c9f0464e07965c05285fd334f0 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:25.707083 4545 tablet_bootstrap.cc:492] T 9acc12c9f0464e07965c05285fd334f0 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap complete.
I20240502 20:46:25.707295 4545 ts_tablet_manager.cc:1367] T 9acc12c9f0464e07965c05285fd334f0 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.024s user 0.023s sys 0.000s
I20240502 20:46:25.707580 29040 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 035f0261307340fa88391a8c2fe564cd
I20240502 20:46:25.707819 4545 raft_consensus.cc:348] T 9acc12c9f0464e07965c05285fd334f0 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:25.708015 4545 raft_consensus.cc:729] T 9acc12c9f0464e07965c05285fd334f0 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:25.708127 29295 tablet_service.cc:1514] Processing DeleteTablet for tablet 035f0261307340fa88391a8c2fe564cd with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:25 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:25.708304 31605 tablet_service.cc:1514] Processing DeleteTablet for tablet 035f0261307340fa88391a8c2fe564cd with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:25 UTC) from {username='slave'} at 127.0.0.1:58346
I20240502 20:46:25.708213 4545 consensus_queue.cc:260] T 9acc12c9f0464e07965c05285fd334f0 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:25.708693 4545 ts_tablet_manager.cc:1398] T 9acc12c9f0464e07965c05285fd334f0 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:25.708757 31475 tablet_service.cc:1514] Processing DeleteTablet for tablet 035f0261307340fa88391a8c2fe564cd with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:25 UTC) from {username='slave'} at 127.0.0.1:40276
I20240502 20:46:25.709187 29597 tablet_copy_service.cc:342] P db17f26a73f3447b92aa9a947e3a7057: Request end of tablet copy session e7ab6b9053664ee5aa71c7978e26fe6a-9acc12c9f0464e07965c05285fd334f0 received from {username='slave'} at 127.22.45.71:51575
I20240502 20:46:25.709256 29597 tablet_copy_service.cc:433] P db17f26a73f3447b92aa9a947e3a7057: ending tablet copy session e7ab6b9053664ee5aa71c7978e26fe6a-9acc12c9f0464e07965c05285fd334f0 on tablet 9acc12c9f0464e07965c05285fd334f0 with peer e7ab6b9053664ee5aa71c7978e26fe6a
I20240502 20:46:25.710093 4551 tablet_replica.cc:330] T 035f0261307340fa88391a8c2fe564cd P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:25.710247 4551 raft_consensus.cc:2229] T 035f0261307340fa88391a8c2fe564cd P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:25.710381 4551 raft_consensus.cc:2258] T 035f0261307340fa88391a8c2fe564cd P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:25.710660 4553 tablet_replica.cc:330] T 035f0261307340fa88391a8c2fe564cd P 2a9c1af8cf814030bde56b67ca5c2fbe: stopping tablet replica
I20240502 20:46:25.711289 4551 ts_tablet_manager.cc:1875] T 035f0261307340fa88391a8c2fe564cd P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:25.712980 4553 raft_consensus.cc:2229] T 035f0261307340fa88391a8c2fe564cd P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:25.713167 4553 raft_consensus.cc:2258] T 035f0261307340fa88391a8c2fe564cd P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:25.714627 4551 ts_tablet_manager.cc:1888] T 035f0261307340fa88391a8c2fe564cd P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:25.714748 4551 log.cc:1196] T 035f0261307340fa88391a8c2fe564cd P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/035f0261307340fa88391a8c2fe564cd
I20240502 20:46:25.715092 4551 ts_tablet_manager.cc:1909] T 035f0261307340fa88391a8c2fe564cd P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:25.715764 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet 035f0261307340fa88391a8c2fe564cd (table rebalancer_dynamic_table_new_name [id=5d82e87b345646069e03d8f5b37ec573]) successfully deleted
I20240502 20:46:25.718854 4553 ts_tablet_manager.cc:1875] T 035f0261307340fa88391a8c2fe564cd P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:25.721021 4553 ts_tablet_manager.cc:1888] T 035f0261307340fa88391a8c2fe564cd P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:25.721135 4553 log.cc:1196] T 035f0261307340fa88391a8c2fe564cd P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/wals/035f0261307340fa88391a8c2fe564cd
I20240502 20:46:25.721386 4553 ts_tablet_manager.cc:1909] T 035f0261307340fa88391a8c2fe564cd P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting consensus metadata
I20240502 20:46:25.721935 4554 ts_tablet_manager.cc:902] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe: Initiating tablet copy from peer dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:25.721920 29028 catalog_manager.cc:4830] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): tablet 035f0261307340fa88391a8c2fe564cd (table rebalancer_dynamic_table_new_name [id=5d82e87b345646069e03d8f5b37ec573]) successfully deleted
I20240502 20:46:25.722370 4554 tablet_copy_client.cc:308] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.67:44095
I20240502 20:46:25.722726 4552 tablet_replica.cc:330] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d: stopping tablet replica
I20240502 20:46:25.722818 29465 tablet_copy_service.cc:140] P dbcdc6ac635b44b185c3eba63f66e9d6: Received BeginTabletCopySession request for tablet 21ee3d7c6f764d0483b335da40e61efc from peer 2a9c1af8cf814030bde56b67ca5c2fbe ({username='slave'} at 127.22.45.69:35145)
I20240502 20:46:25.722916 29465 tablet_copy_service.cc:161] P dbcdc6ac635b44b185c3eba63f66e9d6: Beginning new tablet copy session on tablet 21ee3d7c6f764d0483b335da40e61efc from peer 2a9c1af8cf814030bde56b67ca5c2fbe at {username='slave'} at 127.22.45.69:35145: session id = 2a9c1af8cf814030bde56b67ca5c2fbe-21ee3d7c6f764d0483b335da40e61efc
I20240502 20:46:25.723008 4552 raft_consensus.cc:2229] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:25.723527 4552 raft_consensus.cc:2258] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:25.723790 29465 tablet_copy_source_session.cc:205] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:25.724066 4552 ts_tablet_manager.cc:1875] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:25.724592 4554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21ee3d7c6f764d0483b335da40e61efc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:25.726011 4552 ts_tablet_manager.cc:1888] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:25.726073 4552 log.cc:1196] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal/wals/035f0261307340fa88391a8c2fe564cd
I20240502 20:46:25.726359 4552 ts_tablet_manager.cc:1909] T 035f0261307340fa88391a8c2fe564cd P 315915efcf2b413ca7f9d874338e1b2d: Deleting consensus metadata
I20240502 20:46:25.726423 4554 tablet_copy_client.cc:783] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:25.726603 4554 tablet_copy_client.cc:647] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:25.727015 29027 catalog_manager.cc:4830] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): tablet 035f0261307340fa88391a8c2fe564cd (table rebalancer_dynamic_table_new_name [id=5d82e87b345646069e03d8f5b37ec573]) successfully deleted
I20240502 20:46:25.727617 4554 tablet_copy_client.cc:523] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:25.728643 4554 tablet_bootstrap.cc:492] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:25.751262 4554 tablet_bootstrap.cc:492] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:25.751478 4554 tablet_bootstrap.cc:492] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap complete.
I20240502 20:46:25.751619 4554 ts_tablet_manager.cc:1367] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.023s user 0.017s sys 0.003s
I20240502 20:46:25.751935 4554 raft_consensus.cc:348] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:25.752025 4554 raft_consensus.cc:729] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:25.752138 4554 consensus_queue.cc:260] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:25.752398 4554 ts_tablet_manager.cc:1398] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:25.752858 29465 tablet_copy_service.cc:342] P dbcdc6ac635b44b185c3eba63f66e9d6: Request end of tablet copy session 2a9c1af8cf814030bde56b67ca5c2fbe-21ee3d7c6f764d0483b335da40e61efc received from {username='slave'} at 127.22.45.69:35145
I20240502 20:46:25.752946 29465 tablet_copy_service.cc:433] P dbcdc6ac635b44b185c3eba63f66e9d6: ending tablet copy session 2a9c1af8cf814030bde56b67ca5c2fbe-21ee3d7c6f764d0483b335da40e61efc on tablet 21ee3d7c6f764d0483b335da40e61efc with peer 2a9c1af8cf814030bde56b67ca5c2fbe
I20240502 20:46:25.811846 29042 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:25.816990 29295 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:25.820747 31475 tablet_service.cc:1467] Processing CreateTablet for tablet 806d2c36ccea465dbedb4455acb7ed3c (DEFAULT_TABLE table=rebalancer_dynamic_table [id=4bd76798cf6143869be3b49aceb806ee]), partition=
I20240502 20:46:25.820982 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 806d2c36ccea465dbedb4455acb7ed3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:25.821493 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 806d2c36ccea465dbedb4455acb7ed3c (DEFAULT_TABLE table=rebalancer_dynamic_table [id=4bd76798cf6143869be3b49aceb806ee]), partition=
I20240502 20:46:25.821692 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 806d2c36ccea465dbedb4455acb7ed3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:25.823297 31605 tablet_service.cc:1467] Processing CreateTablet for tablet 806d2c36ccea465dbedb4455acb7ed3c (DEFAULT_TABLE table=rebalancer_dynamic_table [id=4bd76798cf6143869be3b49aceb806ee]), partition=
I20240502 20:46:25.823518 31605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 806d2c36ccea465dbedb4455acb7ed3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:25.823837 4450 tablet_bootstrap.cc:492] T 806d2c36ccea465dbedb4455acb7ed3c P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:25.824709 4450 tablet_bootstrap.cc:654] T 806d2c36ccea465dbedb4455acb7ed3c P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:25.825624 31735 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:25.825842 3845 tablet_bootstrap.cc:492] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:25.826042 4450 tablet_bootstrap.cc:492] T 806d2c36ccea465dbedb4455acb7ed3c P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:25.826115 4450 ts_tablet_manager.cc:1367] T 806d2c36ccea465dbedb4455acb7ed3c P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 20:46:25.826623 4450 raft_consensus.cc:348] T 806d2c36ccea465dbedb4455acb7ed3c P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:25.826721 4450 raft_consensus.cc:374] T 806d2c36ccea465dbedb4455acb7ed3c P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:25.826735 3845 tablet_bootstrap.cc:654] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:25.826750 4450 raft_consensus.cc:729] T 806d2c36ccea465dbedb4455acb7ed3c P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:25.826861 4450 consensus_queue.cc:260] T 806d2c36ccea465dbedb4455acb7ed3c P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:25.827136 4450 ts_tablet_manager.cc:1398] T 806d2c36ccea465dbedb4455acb7ed3c P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:25.827292 3773 tablet_bootstrap.cc:492] T 806d2c36ccea465dbedb4455acb7ed3c P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:25.828115 3773 tablet_bootstrap.cc:654] T 806d2c36ccea465dbedb4455acb7ed3c P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:25.829646 29424 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:25.830050 31475 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:25.833302 31626 raft_consensus.cc:1203] T 3c3bdc0b9e934ae2b81317a371817563 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:25.839979 3773 tablet_bootstrap.cc:492] T 806d2c36ccea465dbedb4455acb7ed3c P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:25.840065 3773 ts_tablet_manager.cc:1367] T 806d2c36ccea465dbedb4455acb7ed3c P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20240502 20:46:25.840480 3773 raft_consensus.cc:348] T 806d2c36ccea465dbedb4455acb7ed3c P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:25.840590 3773 raft_consensus.cc:374] T 806d2c36ccea465dbedb4455acb7ed3c P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:25.840618 3773 raft_consensus.cc:729] T 806d2c36ccea465dbedb4455acb7ed3c P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:25.840718 3773 consensus_queue.cc:260] T 806d2c36ccea465dbedb4455acb7ed3c P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:25.840967 3773 ts_tablet_manager.cc:1398] T 806d2c36ccea465dbedb4455acb7ed3c P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:25.843122 31867 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:25.845657 3845 tablet_bootstrap.cc:492] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:25.845741 3845 ts_tablet_manager.cc:1367] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.000s
I20240502 20:46:25.846163 3845 raft_consensus.cc:348] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:25.846258 3845 raft_consensus.cc:374] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:25.846288 3845 raft_consensus.cc:729] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:25.846395 3845 consensus_queue.cc:260] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:25.846700 3845 ts_tablet_manager.cc:1398] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:25.847620 29163 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:25.853025 31605 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:25.862617 29557 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:25.901276 3829 raft_consensus.cc:482] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:25.901386 3829 raft_consensus.cc:504] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:25.901713 3829 leader_election.cc:288] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:25.901796 3829 consensus_queue.cc:1035] T b699c00ff92b41e9ae24e5982ced3ef9 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:25.902091 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "806d2c36ccea465dbedb4455acb7ed3c" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:25.902212 31888 raft_consensus.cc:2454] T 806d2c36ccea465dbedb4455acb7ed3c P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:25.902487 31429 leader_election.cc:302] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:25.902853 3167 raft_consensus.cc:2781] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:25.902915 3167 raft_consensus.cc:482] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:25.902972 3167 raft_consensus.cc:3037] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:25.903760 3167 raft_consensus.cc:504] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:25.904084 3167 leader_election.cc:288] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Requested vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:25.904433 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "806d2c36ccea465dbedb4455acb7ed3c" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:25.904486 31626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "806d2c36ccea465dbedb4455acb7ed3c" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:25.904542 31888 raft_consensus.cc:3037] T 806d2c36ccea465dbedb4455acb7ed3c P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:25.904592 31626 raft_consensus.cc:2454] T 806d2c36ccea465dbedb4455acb7ed3c P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:25.904631 31624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "806d2c36ccea465dbedb4455acb7ed3c" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:25.904731 31624 raft_consensus.cc:3037] T 806d2c36ccea465dbedb4455acb7ed3c P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:25.905514 31888 raft_consensus.cc:2454] T 806d2c36ccea465dbedb4455acb7ed3c P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:25.905777 31624 raft_consensus.cc:2454] T 806d2c36ccea465dbedb4455acb7ed3c P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:25.906029 31429 leader_election.cc:302] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:25.906165 3167 raft_consensus.cc:2781] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:25.906224 3167 raft_consensus.cc:686] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Becoming Leader. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Running, Role: LEADER
I20240502 20:46:25.906311 3167 consensus_queue.cc:237] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:25.908334 29034 catalog_manager.cc:5484] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69). New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:25.910841 3167 consensus_queue.cc:1035] T 2a00833b5b404e63b0f720b5c8fb528d P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:25.914400 3829 consensus_queue.cc:1035] T b699c00ff92b41e9ae24e5982ced3ef9 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:25.916761 3167 consensus_queue.cc:1035] T 2a00833b5b404e63b0f720b5c8fb528d P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:26.017426 29446 raft_consensus.cc:1203] T 63a41aabb95d4e8a9071c7673b257c47 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 20:46:26.020197 31475 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:26.020345 29295 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:26.020596 31605 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:26.022372 29037 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:26.027065 31624 raft_consensus.cc:1261] T 806d2c36ccea465dbedb4455acb7ed3c P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:26.027132 31888 raft_consensus.cc:1261] T 806d2c36ccea465dbedb4455acb7ed3c P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:26.027619 3167 consensus_queue.cc:1035] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:26.028312 3829 consensus_queue.cc:1035] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:26.030855 4440 tablet.cc:1722] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:26.031154 4440 tablet.cc:1620] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:26.031288 29037 catalog_manager.cc:5576] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207) does not have the latest schema for tablet 806d2c36ccea465dbedb4455acb7ed3c (table rebalancer_dynamic_table [id=4bd76798cf6143869be3b49aceb806ee]). Expected version 1 got 0
I20240502 20:46:26.033305 29037 catalog_manager.cc:5576] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751) does not have the latest schema for tablet 806d2c36ccea465dbedb4455acb7ed3c (table rebalancer_dynamic_table [id=4bd76798cf6143869be3b49aceb806ee]). Expected version 1 got 0
I20240502 20:46:26.033299 4438 tablet.cc:1722] T 806d2c36ccea465dbedb4455acb7ed3c P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:26.033587 4438 tablet.cc:1620] T 806d2c36ccea465dbedb4455acb7ed3c P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:26.038643 31735 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:26.039871 29037 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=4bd76798cf6143869be3b49aceb806ee] alter complete (version 1)
I20240502 20:46:26.040432 4678 tablet.cc:1722] T 806d2c36ccea465dbedb4455acb7ed3c P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:26.040843 4678 tablet.cc:1620] T 806d2c36ccea465dbedb4455acb7ed3c P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:26.048172 29037 catalog_manager.cc:5576] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207) does not have the latest schema for tablet 806d2c36ccea465dbedb4455acb7ed3c (table rebalancer_dynamic_table [id=4bd76798cf6143869be3b49aceb806ee]). Expected version 1 got 0
I20240502 20:46:26.049016 31867 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:26.050400 31757 raft_consensus.cc:1203] T 9acc12c9f0464e07965c05285fd334f0 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:26.054144 29037 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_36"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:26.055451 29557 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:26.067921 29424 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:26.068416 29556 tablet_service.cc:1467] Processing CreateTablet for tablet 08feee781db04883b6dcc030bdaa8afe (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=3e2bf7b49258487aa136e5dbba387a46]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:26.068423 31605 tablet_service.cc:1467] Processing CreateTablet for tablet b9c1ec018c544b3bb90fe57fe9995920 (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=3e2bf7b49258487aa136e5dbba387a46]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:26.068691 31605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9c1ec018c544b3bb90fe57fe9995920. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:26.068692 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08feee781db04883b6dcc030bdaa8afe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:26.068832 31606 tablet_service.cc:1467] Processing CreateTablet for tablet 08feee781db04883b6dcc030bdaa8afe (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=3e2bf7b49258487aa136e5dbba387a46]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:26.069033 31606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08feee781db04883b6dcc030bdaa8afe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:26.070768 31735 tablet_service.cc:1467] Processing CreateTablet for tablet 0f5e277eec15411c8c1fd76189893ee9 (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=3e2bf7b49258487aa136e5dbba387a46]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:26.070750 29295 tablet_service.cc:1467] Processing CreateTablet for tablet b9c1ec018c544b3bb90fe57fe9995920 (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=3e2bf7b49258487aa136e5dbba387a46]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:26.070866 31602 tablet_service.cc:1467] Processing CreateTablet for tablet 0f5e277eec15411c8c1fd76189893ee9 (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=3e2bf7b49258487aa136e5dbba387a46]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:26.071019 31735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f5e277eec15411c8c1fd76189893ee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:26.071040 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9c1ec018c544b3bb90fe57fe9995920. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:26.071053 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f5e277eec15411c8c1fd76189893ee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:26.072917 3773 tablet_bootstrap.cc:492] T 08feee781db04883b6dcc030bdaa8afe P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:26.073050 4680 tablet_bootstrap.cc:492] T b9c1ec018c544b3bb90fe57fe9995920 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:26.073503 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 0f5e277eec15411c8c1fd76189893ee9 (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=3e2bf7b49258487aa136e5dbba387a46]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:26.073735 3773 tablet_bootstrap.cc:654] T 08feee781db04883b6dcc030bdaa8afe P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:26.073741 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f5e277eec15411c8c1fd76189893ee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:26.074554 29557 tablet_service.cc:1467] Processing CreateTablet for tablet b9c1ec018c544b3bb90fe57fe9995920 (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=3e2bf7b49258487aa136e5dbba387a46]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:26.074772 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9c1ec018c544b3bb90fe57fe9995920. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:26.076089 4450 tablet_bootstrap.cc:492] T 0f5e277eec15411c8c1fd76189893ee9 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:26.076968 4450 tablet_bootstrap.cc:654] T 0f5e277eec15411c8c1fd76189893ee9 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:26.077296 31475 tablet_service.cc:1467] Processing CreateTablet for tablet 08feee781db04883b6dcc030bdaa8afe (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=3e2bf7b49258487aa136e5dbba387a46]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:26.077535 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08feee781db04883b6dcc030bdaa8afe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:26.078691 4680 tablet_bootstrap.cc:654] T b9c1ec018c544b3bb90fe57fe9995920 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:26.079521 3845 tablet_bootstrap.cc:492] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:26.079766 4450 tablet_bootstrap.cc:492] T 0f5e277eec15411c8c1fd76189893ee9 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:26.079845 4450 ts_tablet_manager.cc:1367] T 0f5e277eec15411c8c1fd76189893ee9 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240502 20:46:26.080343 3845 tablet_bootstrap.cc:654] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:26.080346 4450 raft_consensus.cc:348] T 0f5e277eec15411c8c1fd76189893ee9 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:26.080441 4450 raft_consensus.cc:374] T 0f5e277eec15411c8c1fd76189893ee9 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:26.080471 4450 raft_consensus.cc:729] T 0f5e277eec15411c8c1fd76189893ee9 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:26.080590 4450 consensus_queue.cc:260] T 0f5e277eec15411c8c1fd76189893ee9 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:26.080842 4450 ts_tablet_manager.cc:1398] T 0f5e277eec15411c8c1fd76189893ee9 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 20:46:26.084051 4681 tablet_bootstrap.cc:492] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:26.084980 4681 tablet_bootstrap.cc:654] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:26.085454 4686 tablet_bootstrap.cc:492] T 08feee781db04883b6dcc030bdaa8afe P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:26.086530 4686 tablet_bootstrap.cc:654] T 08feee781db04883b6dcc030bdaa8afe P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:26.089041 4686 tablet_bootstrap.cc:492] T 08feee781db04883b6dcc030bdaa8afe P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:26.089120 4686 ts_tablet_manager.cc:1367] T 08feee781db04883b6dcc030bdaa8afe P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240502 20:46:26.089525 4686 raft_consensus.cc:348] T 08feee781db04883b6dcc030bdaa8afe P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:26.089622 4686 raft_consensus.cc:374] T 08feee781db04883b6dcc030bdaa8afe P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:26.089653 4686 raft_consensus.cc:729] T 08feee781db04883b6dcc030bdaa8afe P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:26.089759 4686 consensus_queue.cc:260] T 08feee781db04883b6dcc030bdaa8afe P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:26.090042 4686 ts_tablet_manager.cc:1398] T 08feee781db04883b6dcc030bdaa8afe P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:26.090193 4686 tablet_bootstrap.cc:492] T b9c1ec018c544b3bb90fe57fe9995920 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:26.091022 4686 tablet_bootstrap.cc:654] T b9c1ec018c544b3bb90fe57fe9995920 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:26.093627 4686 tablet_bootstrap.cc:492] T b9c1ec018c544b3bb90fe57fe9995920 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:26.093699 4686 ts_tablet_manager.cc:1367] T b9c1ec018c544b3bb90fe57fe9995920 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:26.094113 4686 raft_consensus.cc:348] T b9c1ec018c544b3bb90fe57fe9995920 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:26.094220 4686 raft_consensus.cc:374] T b9c1ec018c544b3bb90fe57fe9995920 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:26.094250 4686 raft_consensus.cc:729] T b9c1ec018c544b3bb90fe57fe9995920 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:26.094298 4680 tablet_bootstrap.cc:492] T b9c1ec018c544b3bb90fe57fe9995920 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:26.094374 4680 ts_tablet_manager.cc:1367] T b9c1ec018c544b3bb90fe57fe9995920 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.000s
I20240502 20:46:26.094355 4686 consensus_queue.cc:260] T b9c1ec018c544b3bb90fe57fe9995920 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:26.094621 4686 ts_tablet_manager.cc:1398] T b9c1ec018c544b3bb90fe57fe9995920 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:26.094910 4680 raft_consensus.cc:348] T b9c1ec018c544b3bb90fe57fe9995920 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:26.095042 4680 raft_consensus.cc:374] T b9c1ec018c544b3bb90fe57fe9995920 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:26.095098 4680 raft_consensus.cc:729] T b9c1ec018c544b3bb90fe57fe9995920 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:26.095222 4680 consensus_queue.cc:260] T b9c1ec018c544b3bb90fe57fe9995920 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:26.095857 4680 ts_tablet_manager.cc:1398] T b9c1ec018c544b3bb90fe57fe9995920 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240502 20:46:26.096563 3845 tablet_bootstrap.cc:492] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:26.096649 3845 ts_tablet_manager.cc:1367] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.000s
I20240502 20:46:26.097069 29163 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:26.097059 3845 raft_consensus.cc:348] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:26.097160 3845 raft_consensus.cc:374] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:26.097188 3845 raft_consensus.cc:729] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:26.097295 3845 consensus_queue.cc:260] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:26.097602 3845 ts_tablet_manager.cc:1398] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:26.099110 3773 tablet_bootstrap.cc:492] T 08feee781db04883b6dcc030bdaa8afe P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:26.099200 3773 ts_tablet_manager.cc:1367] T 08feee781db04883b6dcc030bdaa8afe P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.026s user 0.001s sys 0.000s
I20240502 20:46:26.099619 3773 raft_consensus.cc:348] T 08feee781db04883b6dcc030bdaa8afe P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:26.099743 3773 raft_consensus.cc:374] T 08feee781db04883b6dcc030bdaa8afe P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:26.099783 3773 raft_consensus.cc:729] T 08feee781db04883b6dcc030bdaa8afe P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:26.099896 3773 consensus_queue.cc:260] T 08feee781db04883b6dcc030bdaa8afe P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:26.100163 3773 ts_tablet_manager.cc:1398] T 08feee781db04883b6dcc030bdaa8afe P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:26.100342 3773 tablet_bootstrap.cc:492] T 0f5e277eec15411c8c1fd76189893ee9 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:26.101114 3773 tablet_bootstrap.cc:654] T 0f5e277eec15411c8c1fd76189893ee9 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:26.103266 4681 tablet_bootstrap.cc:492] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:26.103353 4681 ts_tablet_manager.cc:1367] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20240502 20:46:26.103883 4681 raft_consensus.cc:348] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:26.104091 4681 raft_consensus.cc:374] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:26.104127 4681 raft_consensus.cc:729] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:26.104239 4681 consensus_queue.cc:260] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:26.105332 4681 ts_tablet_manager.cc:1398] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:26.109148 4341 consensus_queue.cc:1035] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:26.111794 3773 tablet_bootstrap.cc:492] T 0f5e277eec15411c8c1fd76189893ee9 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:26.111883 3773 ts_tablet_manager.cc:1367] T 0f5e277eec15411c8c1fd76189893ee9 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20240502 20:46:26.112301 3773 raft_consensus.cc:348] T 0f5e277eec15411c8c1fd76189893ee9 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:26.112402 3773 raft_consensus.cc:374] T 0f5e277eec15411c8c1fd76189893ee9 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:26.112434 3773 raft_consensus.cc:729] T 0f5e277eec15411c8c1fd76189893ee9 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:26.112581 3773 consensus_queue.cc:260] T 0f5e277eec15411c8c1fd76189893ee9 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:26.112949 3773 ts_tablet_manager.cc:1398] T 0f5e277eec15411c8c1fd76189893ee9 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:26.113567 3773 tablet_bootstrap.cc:492] T b9c1ec018c544b3bb90fe57fe9995920 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:26.114565 3773 tablet_bootstrap.cc:654] T b9c1ec018c544b3bb90fe57fe9995920 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:26.116083 3773 tablet_bootstrap.cc:492] T b9c1ec018c544b3bb90fe57fe9995920 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:26.116151 3773 ts_tablet_manager.cc:1367] T b9c1ec018c544b3bb90fe57fe9995920 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 20:46:26.116489 3773 raft_consensus.cc:348] T b9c1ec018c544b3bb90fe57fe9995920 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:26.116576 3773 raft_consensus.cc:374] T b9c1ec018c544b3bb90fe57fe9995920 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:26.116601 3773 raft_consensus.cc:729] T b9c1ec018c544b3bb90fe57fe9995920 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:26.116693 3773 consensus_queue.cc:260] T b9c1ec018c544b3bb90fe57fe9995920 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:26.116927 3773 ts_tablet_manager.cc:1398] T b9c1ec018c544b3bb90fe57fe9995920 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:26.123055 4341 consensus_queue.cc:1035] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:26.126523 4216 raft_consensus.cc:482] T b9c1ec018c544b3bb90fe57fe9995920 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:26.126610 4216 raft_consensus.cc:504] T b9c1ec018c544b3bb90fe57fe9995920 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:26.127002 4216 leader_election.cc:288] T b9c1ec018c544b3bb90fe57fe9995920 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:26.127207 31624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9c1ec018c544b3bb90fe57fe9995920" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:26.127322 31624 raft_consensus.cc:2454] T b9c1ec018c544b3bb90fe57fe9995920 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:26.127661 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9c1ec018c544b3bb90fe57fe9995920" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:26.127792 29315 raft_consensus.cc:2454] T b9c1ec018c544b3bb90fe57fe9995920 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:26.127823 29511 leader_election.cc:302] T b9c1ec018c544b3bb90fe57fe9995920 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:26.127970 4216 raft_consensus.cc:2781] T b9c1ec018c544b3bb90fe57fe9995920 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:26.128028 4216 raft_consensus.cc:482] T b9c1ec018c544b3bb90fe57fe9995920 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:26.128062 4216 raft_consensus.cc:3037] T b9c1ec018c544b3bb90fe57fe9995920 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:26.129014 4216 raft_consensus.cc:504] T b9c1ec018c544b3bb90fe57fe9995920 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:26.129325 4216 leader_election.cc:288] T b9c1ec018c544b3bb90fe57fe9995920 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:26.129592 31624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9c1ec018c544b3bb90fe57fe9995920" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:26.129676 31624 raft_consensus.cc:3037] T b9c1ec018c544b3bb90fe57fe9995920 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:26.130359 31624 raft_consensus.cc:2454] T b9c1ec018c544b3bb90fe57fe9995920 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:26.130630 29511 leader_election.cc:302] T b9c1ec018c544b3bb90fe57fe9995920 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:26.130685 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9c1ec018c544b3bb90fe57fe9995920" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:26.130784 4216 raft_consensus.cc:2781] T b9c1ec018c544b3bb90fe57fe9995920 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:26.130806 29315 raft_consensus.cc:3037] T b9c1ec018c544b3bb90fe57fe9995920 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:26.130846 4216 raft_consensus.cc:686] T b9c1ec018c544b3bb90fe57fe9995920 P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Becoming Leader. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Running, Role: LEADER
I20240502 20:46:26.130944 4216 consensus_queue.cc:237] T b9c1ec018c544b3bb90fe57fe9995920 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:26.131637 29315 raft_consensus.cc:2454] T b9c1ec018c544b3bb90fe57fe9995920 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:26.132738 29037 catalog_manager.cc:5484] T b9c1ec018c544b3bb90fe57fe9995920 P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: term changed from 0 to 1, leader changed from <none> to db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:26.147346 29037 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:26.152237 4440 tablet.cc:1722] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:26.152544 4440 tablet.cc:1620] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:26.153144 2289 raft_consensus.cc:482] T 0f5e277eec15411c8c1fd76189893ee9 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:26.153229 2289 raft_consensus.cc:504] T 0f5e277eec15411c8c1fd76189893ee9 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:26.153568 2289 leader_election.cc:288] T 0f5e277eec15411c8c1fd76189893ee9 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:26.153874 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f5e277eec15411c8c1fd76189893ee9" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:26.154007 31757 raft_consensus.cc:2454] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:26.154425 31559 leader_election.cc:302] T 0f5e277eec15411c8c1fd76189893ee9 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:26.154817 2289 raft_consensus.cc:2781] T 0f5e277eec15411c8c1fd76189893ee9 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:26.154875 4438 tablet.cc:1722] T 806d2c36ccea465dbedb4455acb7ed3c P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:26.155160 4438 tablet.cc:1620] T 806d2c36ccea465dbedb4455acb7ed3c P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:26.155153 4678 tablet.cc:1722] T 806d2c36ccea465dbedb4455acb7ed3c P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:26.155426 4678 tablet.cc:1620] T 806d2c36ccea465dbedb4455acb7ed3c P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:26.156067 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f5e277eec15411c8c1fd76189893ee9" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:26.156183 31888 raft_consensus.cc:2454] T 0f5e277eec15411c8c1fd76189893ee9 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:26.154877 2289 raft_consensus.cc:482] T 0f5e277eec15411c8c1fd76189893ee9 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:26.157294 2289 raft_consensus.cc:3037] T 0f5e277eec15411c8c1fd76189893ee9 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:26.158078 2289 raft_consensus.cc:504] T 0f5e277eec15411c8c1fd76189893ee9 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:26.158389 2289 leader_election.cc:288] T 0f5e277eec15411c8c1fd76189893ee9 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Requested vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:26.158926 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f5e277eec15411c8c1fd76189893ee9" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:26.159040 31888 raft_consensus.cc:3037] T 0f5e277eec15411c8c1fd76189893ee9 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:26.159539 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f5e277eec15411c8c1fd76189893ee9" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:26.159639 31757 raft_consensus.cc:3037] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:26.159945 31888 raft_consensus.cc:2454] T 0f5e277eec15411c8c1fd76189893ee9 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:26.160255 31560 leader_election.cc:302] T 0f5e277eec15411c8c1fd76189893ee9 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:26.160274 29037 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=4bd76798cf6143869be3b49aceb806ee] alter complete (version 2)
I20240502 20:46:26.160406 2289 raft_consensus.cc:2781] T 0f5e277eec15411c8c1fd76189893ee9 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:26.160471 2289 raft_consensus.cc:686] T 0f5e277eec15411c8c1fd76189893ee9 P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Becoming Leader. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Running, Role: LEADER
I20240502 20:46:26.160604 31757 raft_consensus.cc:2454] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:26.160590 2289 consensus_queue.cc:237] T 0f5e277eec15411c8c1fd76189893ee9 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:26.165390 29041 catalog_manager.cc:5484] T 0f5e277eec15411c8c1fd76189893ee9 P 315915efcf2b413ca7f9d874338e1b2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70). New cstate: current_term: 1 leader_uuid: "315915efcf2b413ca7f9d874338e1b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:26.168849 31494 consensus_queue.cc:237] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:26.169957 29315 raft_consensus.cc:1261] T 63a41aabb95d4e8a9071c7673b257c47 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:26.169994 29446 raft_consensus.cc:1261] T 63a41aabb95d4e8a9071c7673b257c47 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:26.170295 3829 consensus_queue.cc:1035] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:26.170444 3167 consensus_queue.cc:1035] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:26.172087 4619 raft_consensus.cc:2932] T 63a41aabb95d4e8a9071c7673b257c47 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:26.172528 29446 raft_consensus.cc:2932] T 63a41aabb95d4e8a9071c7673b257c47 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:26.173987 29315 raft_consensus.cc:2932] T 63a41aabb95d4e8a9071c7673b257c47 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:26.175522 29042 catalog_manager.cc:5484] T 63a41aabb95d4e8a9071c7673b257c47 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index 2 to 3, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) evicted. New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:26.179545 31735 tablet_service.cc:1514] Processing DeleteTablet for tablet 63a41aabb95d4e8a9071c7673b257c47 with delete_type TABLET_DATA_TOMBSTONED (TS e7ab6b9053664ee5aa71c7978e26fe6a not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:33864
I20240502 20:46:26.179836 4717 tablet_replica.cc:330] T 63a41aabb95d4e8a9071c7673b257c47 P e7ab6b9053664ee5aa71c7978e26fe6a: stopping tablet replica
I20240502 20:46:26.179975 4717 raft_consensus.cc:2229] T 63a41aabb95d4e8a9071c7673b257c47 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:26.180100 4717 raft_consensus.cc:2258] T 63a41aabb95d4e8a9071c7673b257c47 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:26.180518 4717 ts_tablet_manager.cc:1875] T 63a41aabb95d4e8a9071c7673b257c47 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:26.182621 4717 ts_tablet_manager.cc:1888] T 63a41aabb95d4e8a9071c7673b257c47 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 20:46:26.182693 4717 log.cc:1196] T 63a41aabb95d4e8a9071c7673b257c47 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/wals/63a41aabb95d4e8a9071c7673b257c47
I20240502 20:46:26.183187 29026 catalog_manager.cc:4830] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): tablet 63a41aabb95d4e8a9071c7673b257c47 (table rebalancer_extra_table_4 [id=57203d81e8ab41bcb24361b9b6107024]) successfully deleted
I20240502 20:46:26.202697 31888 raft_consensus.cc:1203] T 28d6a4d9c92141f794d6ee04335cf9b8 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 20:46:26.219714 31494 raft_consensus.cc:1203] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:26.263334 29036 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:26.268335 4440 tablet.cc:1722] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:26.269716 4438 tablet.cc:1722] T 806d2c36ccea465dbedb4455acb7ed3c P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:26.270829 4678 tablet.cc:1722] T 806d2c36ccea465dbedb4455acb7ed3c P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:26.273838 29036 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=4bd76798cf6143869be3b49aceb806ee] alter complete (version 3)
I20240502 20:46:26.297745 3982 raft_consensus.cc:482] T 08feee781db04883b6dcc030bdaa8afe P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:26.297853 3982 raft_consensus.cc:504] T 08feee781db04883b6dcc030bdaa8afe P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:26.298202 3982 leader_election.cc:288] T 08feee781db04883b6dcc030bdaa8afe P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:26.298506 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08feee781db04883b6dcc030bdaa8afe" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:26.298558 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08feee781db04883b6dcc030bdaa8afe" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:26.298640 31494 raft_consensus.cc:2454] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:26.298687 29577 raft_consensus.cc:2454] T 08feee781db04883b6dcc030bdaa8afe P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:26.298888 31561 leader_election.cc:302] T 08feee781db04883b6dcc030bdaa8afe P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, 315915efcf2b413ca7f9d874338e1b2d; no voters:
I20240502 20:46:26.299041 3982 raft_consensus.cc:2781] T 08feee781db04883b6dcc030bdaa8afe P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:26.299101 3982 raft_consensus.cc:482] T 08feee781db04883b6dcc030bdaa8afe P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:26.299137 3982 raft_consensus.cc:3037] T 08feee781db04883b6dcc030bdaa8afe P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:26.299858 3982 raft_consensus.cc:504] T 08feee781db04883b6dcc030bdaa8afe P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:26.300153 3982 leader_election.cc:288] T 08feee781db04883b6dcc030bdaa8afe P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Requested vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:26.300424 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08feee781db04883b6dcc030bdaa8afe" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:26.300444 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08feee781db04883b6dcc030bdaa8afe" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:26.300525 29577 raft_consensus.cc:3037] T 08feee781db04883b6dcc030bdaa8afe P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:26.300534 31494 raft_consensus.cc:3037] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:26.301261 31494 raft_consensus.cc:2454] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:26.301316 29577 raft_consensus.cc:2454] T 08feee781db04883b6dcc030bdaa8afe P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:26.301517 31561 leader_election.cc:302] T 08feee781db04883b6dcc030bdaa8afe P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, 315915efcf2b413ca7f9d874338e1b2d; no voters:
I20240502 20:46:26.301733 3982 raft_consensus.cc:2781] T 08feee781db04883b6dcc030bdaa8afe P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:26.301823 3982 raft_consensus.cc:686] T 08feee781db04883b6dcc030bdaa8afe P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Becoming Leader. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Running, Role: LEADER
I20240502 20:46:26.301949 3982 consensus_queue.cc:237] T 08feee781db04883b6dcc030bdaa8afe P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:26.303915 29033 catalog_manager.cc:5484] T 08feee781db04883b6dcc030bdaa8afe P 315915efcf2b413ca7f9d874338e1b2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70). New cstate: current_term: 1 leader_uuid: "315915efcf2b413ca7f9d874338e1b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:26.328512 29556 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:26.331290 29295 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:26.332628 31475 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:26.336217 31605 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:26.342818 31735 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:26.345907 29163 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:26.348289 29424 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:26.362438 31867 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:26.378966 29033 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:26.379114 29033 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:26.381403 29033 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 806d2c36ccea465dbedb4455acb7ed3c
I20240502 20:46:26.381963 31605 tablet_service.cc:1514] Processing DeleteTablet for tablet 806d2c36ccea465dbedb4455acb7ed3c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:26 UTC) from {username='slave'} at 127.0.0.1:58346
I20240502 20:46:26.382073 31867 tablet_service.cc:1514] Processing DeleteTablet for tablet 806d2c36ccea465dbedb4455acb7ed3c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:26 UTC) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:26.382341 31475 tablet_service.cc:1514] Processing DeleteTablet for tablet 806d2c36ccea465dbedb4455acb7ed3c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:26 UTC) from {username='slave'} at 127.0.0.1:40276
I20240502 20:46:26.382603 4774 tablet_replica.cc:330] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe: stopping tablet replica
I20240502 20:46:26.382722 4774 raft_consensus.cc:2229] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:26.382863 4774 raft_consensus.cc:2258] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:26.383370 4774 ts_tablet_manager.cc:1875] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:26.385465 4774 ts_tablet_manager.cc:1888] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:26.385607 4774 log.cc:1196] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/wals/806d2c36ccea465dbedb4455acb7ed3c
I20240502 20:46:26.385949 4774 ts_tablet_manager.cc:1909] T 806d2c36ccea465dbedb4455acb7ed3c P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting consensus metadata
I20240502 20:46:26.386723 29028 catalog_manager.cc:4830] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): tablet 806d2c36ccea465dbedb4455acb7ed3c (table rebalancer_dynamic_table_new_name [id=4bd76798cf6143869be3b49aceb806ee]) successfully deleted
I20240502 20:46:26.393151 4773 tablet_replica.cc:330] T 806d2c36ccea465dbedb4455acb7ed3c P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:26.393334 4773 raft_consensus.cc:2229] T 806d2c36ccea465dbedb4455acb7ed3c P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:26.393834 4773 raft_consensus.cc:2258] T 806d2c36ccea465dbedb4455acb7ed3c P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:26.394515 4773 ts_tablet_manager.cc:1875] T 806d2c36ccea465dbedb4455acb7ed3c P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:26.402139 4773 ts_tablet_manager.cc:1888] T 806d2c36ccea465dbedb4455acb7ed3c P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:26.402252 4773 log.cc:1196] T 806d2c36ccea465dbedb4455acb7ed3c P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/806d2c36ccea465dbedb4455acb7ed3c
I20240502 20:46:26.402571 4773 ts_tablet_manager.cc:1909] T 806d2c36ccea465dbedb4455acb7ed3c P e6b60803d0764d228c863e2da65e3691: Deleting consensus metadata
I20240502 20:46:26.403308 4772 tablet_replica.cc:330] T 806d2c36ccea465dbedb4455acb7ed3c P 315915efcf2b413ca7f9d874338e1b2d: stopping tablet replica
I20240502 20:46:26.403483 4772 raft_consensus.cc:2229] T 806d2c36ccea465dbedb4455acb7ed3c P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:26.403645 4772 raft_consensus.cc:2258] T 806d2c36ccea465dbedb4455acb7ed3c P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:26.404307 4772 ts_tablet_manager.cc:1875] T 806d2c36ccea465dbedb4455acb7ed3c P 315915efcf2b413ca7f9d874338e1b2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:26.406365 4772 ts_tablet_manager.cc:1888] T 806d2c36ccea465dbedb4455acb7ed3c P 315915efcf2b413ca7f9d874338e1b2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:26.406507 4772 log.cc:1196] T 806d2c36ccea465dbedb4455acb7ed3c P 315915efcf2b413ca7f9d874338e1b2d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal/wals/806d2c36ccea465dbedb4455acb7ed3c
I20240502 20:46:26.406683 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet 806d2c36ccea465dbedb4455acb7ed3c (table rebalancer_dynamic_table_new_name [id=4bd76798cf6143869be3b49aceb806ee]) successfully deleted
I20240502 20:46:26.406842 4772 ts_tablet_manager.cc:1909] T 806d2c36ccea465dbedb4455acb7ed3c P 315915efcf2b413ca7f9d874338e1b2d: Deleting consensus metadata
I20240502 20:46:26.407446 29027 catalog_manager.cc:4830] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): tablet 806d2c36ccea465dbedb4455acb7ed3c (table rebalancer_dynamic_table_new_name [id=4bd76798cf6143869be3b49aceb806ee]) successfully deleted
I20240502 20:46:26.429975 29446 consensus_queue.cc:237] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:26.431133 31624 raft_consensus.cc:1261] T 3c3bdc0b9e934ae2b81317a371817563 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:26.431450 29315 raft_consensus.cc:1261] T 3c3bdc0b9e934ae2b81317a371817563 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:26.431512 4341 consensus_queue.cc:1035] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:26.431959 4341 consensus_queue.cc:1035] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:26.433256 4694 raft_consensus.cc:2932] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:26.433647 31625 raft_consensus.cc:2932] T 3c3bdc0b9e934ae2b81317a371817563 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:26.434101 29315 raft_consensus.cc:2932] T 3c3bdc0b9e934ae2b81317a371817563 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:26.436688 29034 catalog_manager.cc:5484] T 3c3bdc0b9e934ae2b81317a371817563 P 09258ac095664fa99caf0707262cde13 reported cstate change: config changed from index 4 to 5, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) evicted. New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:26.439077 29556 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c3bdc0b9e934ae2b81317a371817563 with delete_type TABLET_DATA_TOMBSTONED (TS db17f26a73f3447b92aa9a947e3a7057 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:26.440970 4793 tablet_replica.cc:330] T 3c3bdc0b9e934ae2b81317a371817563 P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:26.441215 4793 raft_consensus.cc:2229] T 3c3bdc0b9e934ae2b81317a371817563 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:26.441361 4793 raft_consensus.cc:2258] T 3c3bdc0b9e934ae2b81317a371817563 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:26.441767 4793 ts_tablet_manager.cc:1875] T 3c3bdc0b9e934ae2b81317a371817563 P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:26.443934 4793 ts_tablet_manager.cc:1888] T 3c3bdc0b9e934ae2b81317a371817563 P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 20:46:26.444010 4793 log.cc:1196] T 3c3bdc0b9e934ae2b81317a371817563 P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/3c3bdc0b9e934ae2b81317a371817563
I20240502 20:46:26.444521 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet 3c3bdc0b9e934ae2b81317a371817563 (table rebalance_test_table_11 [id=cc807ad977d146dc84be7ee2897657fc]) successfully deleted
I20240502 20:46:26.484189 29033 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:26.489415 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 8db888c800cf41ddb08b3d38180aacef (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c16a44861d9a4e0f9abf52b84940f2c7]), partition=
I20240502 20:46:26.489609 29163 tablet_service.cc:1467] Processing CreateTablet for tablet 8db888c800cf41ddb08b3d38180aacef (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c16a44861d9a4e0f9abf52b84940f2c7]), partition=
I20240502 20:46:26.489683 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8db888c800cf41ddb08b3d38180aacef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:26.489836 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8db888c800cf41ddb08b3d38180aacef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:26.491942 4450 tablet_bootstrap.cc:492] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:26.492276 31605 tablet_service.cc:1467] Processing CreateTablet for tablet 8db888c800cf41ddb08b3d38180aacef (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c16a44861d9a4e0f9abf52b84940f2c7]), partition=
I20240502 20:46:26.492494 31605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8db888c800cf41ddb08b3d38180aacef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:26.492599 4836 tablet_bootstrap.cc:492] T 8db888c800cf41ddb08b3d38180aacef P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:26.492851 4450 tablet_bootstrap.cc:654] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:26.494423 4450 tablet_bootstrap.cc:492] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:26.494526 4450 ts_tablet_manager.cc:1367] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 20:46:26.495002 4450 raft_consensus.cc:348] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:26.495100 4450 raft_consensus.cc:374] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:26.495131 4450 raft_consensus.cc:729] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:26.495260 4450 consensus_queue.cc:260] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:26.495548 4450 ts_tablet_manager.cc:1398] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:26.500016 4836 tablet_bootstrap.cc:654] T 8db888c800cf41ddb08b3d38180aacef P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:26.504058 3773 tablet_bootstrap.cc:492] T 8db888c800cf41ddb08b3d38180aacef P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:26.504994 3773 tablet_bootstrap.cc:654] T 8db888c800cf41ddb08b3d38180aacef P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:26.507118 4836 tablet_bootstrap.cc:492] T 8db888c800cf41ddb08b3d38180aacef P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:26.507241 4836 ts_tablet_manager.cc:1367] T 8db888c800cf41ddb08b3d38180aacef P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 20:46:26.507707 4836 raft_consensus.cc:348] T 8db888c800cf41ddb08b3d38180aacef P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:26.507848 4836 raft_consensus.cc:374] T 8db888c800cf41ddb08b3d38180aacef P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:26.507899 4836 raft_consensus.cc:729] T 8db888c800cf41ddb08b3d38180aacef P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:26.508030 4836 consensus_queue.cc:260] T 8db888c800cf41ddb08b3d38180aacef P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:26.508919 3773 tablet_bootstrap.cc:492] T 8db888c800cf41ddb08b3d38180aacef P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:26.508963 4836 ts_tablet_manager.cc:1398] T 8db888c800cf41ddb08b3d38180aacef P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:26.509008 3773 ts_tablet_manager.cc:1367] T 8db888c800cf41ddb08b3d38180aacef P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240502 20:46:26.509361 3773 raft_consensus.cc:348] T 8db888c800cf41ddb08b3d38180aacef P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:26.509436 3773 raft_consensus.cc:374] T 8db888c800cf41ddb08b3d38180aacef P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:26.509455 3773 raft_consensus.cc:729] T 8db888c800cf41ddb08b3d38180aacef P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:26.509528 3773 consensus_queue.cc:260] T 8db888c800cf41ddb08b3d38180aacef P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:26.509747 3773 ts_tablet_manager.cc:1398] T 8db888c800cf41ddb08b3d38180aacef P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:26.540721 29295 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:26.548791 29163 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:26.550633 31475 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:26.558629 31867 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:26.572021 31735 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:26.576094 31605 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:26.578420 29556 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:26.581566 29424 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:26.587777 4679 raft_consensus.cc:482] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:26.587877 4679 raft_consensus.cc:504] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:26.588244 4679 leader_election.cc:288] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:26.588588 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8db888c800cf41ddb08b3d38180aacef" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:26.588711 31625 raft_consensus.cc:2454] T 8db888c800cf41ddb08b3d38180aacef P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:26.588945 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8db888c800cf41ddb08b3d38180aacef" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:26.588997 31822 leader_election.cc:302] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:26.589077 29175 raft_consensus.cc:2454] T 8db888c800cf41ddb08b3d38180aacef P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:26.589152 4679 raft_consensus.cc:2781] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:26.589190 4679 raft_consensus.cc:482] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:26.589219 4679 raft_consensus.cc:3037] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:26.590277 4679 raft_consensus.cc:504] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:26.590631 4679 leader_election.cc:288] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:26.591042 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8db888c800cf41ddb08b3d38180aacef" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:26.591146 31625 raft_consensus.cc:3037] T 8db888c800cf41ddb08b3d38180aacef P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:26.592072 31625 raft_consensus.cc:2454] T 8db888c800cf41ddb08b3d38180aacef P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:26.592352 31822 leader_election.cc:302] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:26.592564 4679 raft_consensus.cc:2781] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:26.592636 4679 raft_consensus.cc:686] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Becoming Leader. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Running, Role: LEADER
I20240502 20:46:26.592746 4679 consensus_queue.cc:237] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:26.592855 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8db888c800cf41ddb08b3d38180aacef" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:26.592952 29175 raft_consensus.cc:3037] T 8db888c800cf41ddb08b3d38180aacef P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:26.593863 29175 raft_consensus.cc:2454] T 8db888c800cf41ddb08b3d38180aacef P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:26.595384 29037 catalog_manager.cc:5484] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6b60803d0764d228c863e2da65e3691 (127.22.45.72). New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:26.611511 2289 consensus_queue.cc:1035] T 0f5e277eec15411c8c1fd76189893ee9 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:26.624259 2289 consensus_queue.cc:1035] T 0f5e277eec15411c8c1fd76189893ee9 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:26.647511 31625 consensus_queue.cc:237] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:26.648610 31755 raft_consensus.cc:1261] T 90bab3d11d424b3abcdc74d5da51cbee P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer 315915efcf2b413ca7f9d874338e1b2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:26.648654 29446 raft_consensus.cc:1261] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 315915efcf2b413ca7f9d874338e1b2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:26.648988 4852 consensus_queue.cc:1035] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:26.649183 2289 consensus_queue.cc:1035] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:26.651109 2289 raft_consensus.cc:2932] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:26.652315 31755 raft_consensus.cc:2932] T 90bab3d11d424b3abcdc74d5da51cbee P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:26.653592 29037 catalog_manager.cc:5484] T 90bab3d11d424b3abcdc74d5da51cbee P 315915efcf2b413ca7f9d874338e1b2d reported cstate change: config changed from index 2 to 3, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) evicted. New cstate: current_term: 1 leader_uuid: "315915efcf2b413ca7f9d874338e1b2d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:26.654013 29446 raft_consensus.cc:2932] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:26.657298 31867 tablet_service.cc:1514] Processing DeleteTablet for tablet 90bab3d11d424b3abcdc74d5da51cbee with delete_type TABLET_DATA_TOMBSTONED (TS e6b60803d0764d228c863e2da65e3691 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:26.657533 4773 tablet_replica.cc:330] T 90bab3d11d424b3abcdc74d5da51cbee P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:26.657685 4773 raft_consensus.cc:2229] T 90bab3d11d424b3abcdc74d5da51cbee P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:26.657812 4773 raft_consensus.cc:2258] T 90bab3d11d424b3abcdc74d5da51cbee P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:26.658255 4773 ts_tablet_manager.cc:1875] T 90bab3d11d424b3abcdc74d5da51cbee P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:26.660246 4773 ts_tablet_manager.cc:1888] T 90bab3d11d424b3abcdc74d5da51cbee P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 20:46:26.660295 4773 log.cc:1196] T 90bab3d11d424b3abcdc74d5da51cbee P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/90bab3d11d424b3abcdc74d5da51cbee
I20240502 20:46:26.660686 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet 90bab3d11d424b3abcdc74d5da51cbee (table rebalancer_extra_table_2 [id=536c055ebf024097b825854cd0490c67]) successfully deleted
I20240502 20:46:26.702617 4216 consensus_queue.cc:1035] T b9c1ec018c544b3bb90fe57fe9995920 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:26.708190 4915 consensus_queue.cc:1035] T b9c1ec018c544b3bb90fe57fe9995920 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:26.709430 3982 consensus_queue.cc:1035] T 08feee781db04883b6dcc030bdaa8afe P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:26.712939 2289 consensus_queue.cc:1035] T 08feee781db04883b6dcc030bdaa8afe P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:26.714373 29034 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:26.719861 31625 raft_consensus.cc:1261] T 8db888c800cf41ddb08b3d38180aacef P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:26.720045 29175 raft_consensus.cc:1261] T 8db888c800cf41ddb08b3d38180aacef P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:26.720400 4679 consensus_queue.cc:1035] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:26.720690 4329 consensus_queue.cc:1035] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:26.722947 29042 catalog_manager.cc:5576] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751) does not have the latest schema for tablet 8db888c800cf41ddb08b3d38180aacef (table rebalancer_dynamic_table [id=c16a44861d9a4e0f9abf52b84940f2c7]). Expected version 1 got 0
I20240502 20:46:26.723668 29033 catalog_manager.cc:5576] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267) does not have the latest schema for tablet 8db888c800cf41ddb08b3d38180aacef (table rebalancer_dynamic_table [id=c16a44861d9a4e0f9abf52b84940f2c7]). Expected version 1 got 0
I20240502 20:46:26.725025 4621 mvcc.cc:205] Tried to move back new op lower bound from 7023340694400614400 to 7023340693886889984. Current Snapshot: MvccSnapshot[applied={T|T < 7023340694400614400}]
I20240502 20:46:26.725263 4678 tablet.cc:1722] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:26.725561 4678 tablet.cc:1620] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:26.727507 4438 tablet.cc:1722] T 8db888c800cf41ddb08b3d38180aacef P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:26.727777 4438 tablet.cc:1620] T 8db888c800cf41ddb08b3d38180aacef P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:26.729017 29033 catalog_manager.cc:5576] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267) does not have the latest schema for tablet 8db888c800cf41ddb08b3d38180aacef (table rebalancer_dynamic_table [id=c16a44861d9a4e0f9abf52b84940f2c7]). Expected version 1 got 0
I20240502 20:46:26.731660 4944 tablet.cc:1722] T 8db888c800cf41ddb08b3d38180aacef P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:26.731922 4944 tablet.cc:1620] T 8db888c800cf41ddb08b3d38180aacef P 7796ea3fba294d17a34386873969431d: MemRowSet was empty: no flush needed.
I20240502 20:46:26.735255 29036 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=c16a44861d9a4e0f9abf52b84940f2c7] alter complete (version 1)
I20240502 20:46:26.780231 31475 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:26.780570 29295 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 20:46:26.785722 31605 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:26.786146 29163 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:26.786324 31735 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:26.792065 29424 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:26.810619 31867 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:26.811053 29556 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:26.836881 29033 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_38"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:26.840447 29036 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:26.848881 4678 tablet.cc:1722] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:26.849195 4678 tablet.cc:1620] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:26.849231 4438 tablet.cc:1722] T 8db888c800cf41ddb08b3d38180aacef P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:26.849505 4438 tablet.cc:1620] T 8db888c800cf41ddb08b3d38180aacef P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:26.850086 4944 tablet.cc:1722] T 8db888c800cf41ddb08b3d38180aacef P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:26.850382 4944 tablet.cc:1620] T 8db888c800cf41ddb08b3d38180aacef P 7796ea3fba294d17a34386873969431d: MemRowSet was empty: no flush needed.
I20240502 20:46:26.850612 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 27bb5cc8379f44f49a9f7d1cc6ae1311 (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=a84b1ec00cc44d43a5c83be915091509]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:26.850888 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27bb5cc8379f44f49a9f7d1cc6ae1311. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:26.851295 31735 tablet_service.cc:1467] Processing CreateTablet for tablet 361b41488c5347e0aef692a39cd347c4 (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=a84b1ec00cc44d43a5c83be915091509]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:26.851522 31735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 361b41488c5347e0aef692a39cd347c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:26.852980 4450 tablet_bootstrap.cc:492] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:26.853847 4450 tablet_bootstrap.cc:654] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:26.854367 29036 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=c16a44861d9a4e0f9abf52b84940f2c7] alter complete (version 2)
I20240502 20:46:26.854710 4957 tablet_bootstrap.cc:492] T 361b41488c5347e0aef692a39cd347c4 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:26.854805 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 27bb5cc8379f44f49a9f7d1cc6ae1311 (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=a84b1ec00cc44d43a5c83be915091509]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:26.854895 29556 tablet_service.cc:1467] Processing CreateTablet for tablet 361b41488c5347e0aef692a39cd347c4 (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=a84b1ec00cc44d43a5c83be915091509]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:26.855041 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27bb5cc8379f44f49a9f7d1cc6ae1311. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:26.855166 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 361b41488c5347e0aef692a39cd347c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:26.855175 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 60e8272de78a4c94b77408c4d1d5f60e (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=a84b1ec00cc44d43a5c83be915091509]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:26.855365 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60e8272de78a4c94b77408c4d1d5f60e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:26.855530 4957 tablet_bootstrap.cc:654] T 361b41488c5347e0aef692a39cd347c4 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:26.858016 4960 tablet_bootstrap.cc:492] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:26.858187 31475 tablet_service.cc:1467] Processing CreateTablet for tablet 60e8272de78a4c94b77408c4d1d5f60e (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=a84b1ec00cc44d43a5c83be915091509]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:26.858438 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60e8272de78a4c94b77408c4d1d5f60e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:26.858894 4960 tablet_bootstrap.cc:654] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:26.860121 4960 tablet_bootstrap.cc:492] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:26.860203 4960 ts_tablet_manager.cc:1367] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:26.860755 4960 raft_consensus.cc:348] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:26.860960 4960 raft_consensus.cc:374] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:26.861044 4960 raft_consensus.cc:729] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:26.861089 29426 tablet_service.cc:1467] Processing CreateTablet for tablet 361b41488c5347e0aef692a39cd347c4 (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=a84b1ec00cc44d43a5c83be915091509]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:26.861217 4960 consensus_queue.cc:260] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:26.861294 29425 tablet_service.cc:1467] Processing CreateTablet for tablet 60e8272de78a4c94b77408c4d1d5f60e (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=a84b1ec00cc44d43a5c83be915091509]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:26.861491 4960 ts_tablet_manager.cc:1398] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:26.861106 29424 tablet_service.cc:1467] Processing CreateTablet for tablet 27bb5cc8379f44f49a9f7d1cc6ae1311 (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=a84b1ec00cc44d43a5c83be915091509]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:26.862329 4963 tablet_bootstrap.cc:492] T 60e8272de78a4c94b77408c4d1d5f60e P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:26.863034 4957 tablet_bootstrap.cc:492] T 361b41488c5347e0aef692a39cd347c4 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:26.861344 29426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 361b41488c5347e0aef692a39cd347c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:26.863235 4957 ts_tablet_manager.cc:1367] T 361b41488c5347e0aef692a39cd347c4 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 20:46:26.863256 4963 tablet_bootstrap.cc:654] T 60e8272de78a4c94b77408c4d1d5f60e P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:26.863716 4957 raft_consensus.cc:348] T 361b41488c5347e0aef692a39cd347c4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:26.863876 4957 raft_consensus.cc:374] T 361b41488c5347e0aef692a39cd347c4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:26.863956 4957 raft_consensus.cc:729] T 361b41488c5347e0aef692a39cd347c4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:26.864156 4957 consensus_queue.cc:260] T 361b41488c5347e0aef692a39cd347c4 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:26.865177 4957 ts_tablet_manager.cc:1398] T 361b41488c5347e0aef692a39cd347c4 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:26.865208 4965 tablet_bootstrap.cc:492] T 361b41488c5347e0aef692a39cd347c4 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:26.865228 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60e8272de78a4c94b77408c4d1d5f60e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:26.866225 4965 tablet_bootstrap.cc:654] T 361b41488c5347e0aef692a39cd347c4 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:26.866570 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27bb5cc8379f44f49a9f7d1cc6ae1311. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:26.867383 4965 tablet_bootstrap.cc:492] T 361b41488c5347e0aef692a39cd347c4 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:26.867458 4965 ts_tablet_manager.cc:1367] T 361b41488c5347e0aef692a39cd347c4 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:26.867846 4965 raft_consensus.cc:348] T 361b41488c5347e0aef692a39cd347c4 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:26.867931 4965 raft_consensus.cc:374] T 361b41488c5347e0aef692a39cd347c4 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:26.867955 4965 raft_consensus.cc:729] T 361b41488c5347e0aef692a39cd347c4 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:26.868057 4965 consensus_queue.cc:260] T 361b41488c5347e0aef692a39cd347c4 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:26.868283 4965 ts_tablet_manager.cc:1398] T 361b41488c5347e0aef692a39cd347c4 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:26.868404 4965 tablet_bootstrap.cc:492] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:26.869143 4965 tablet_bootstrap.cc:654] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:26.874545 4961 tablet_bootstrap.cc:492] T 361b41488c5347e0aef692a39cd347c4 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:26.875152 4963 tablet_bootstrap.cc:492] T 60e8272de78a4c94b77408c4d1d5f60e P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:26.875242 4963 ts_tablet_manager.cc:1367] T 60e8272de78a4c94b77408c4d1d5f60e P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 20:46:26.875640 4961 tablet_bootstrap.cc:654] T 361b41488c5347e0aef692a39cd347c4 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:26.875674 4963 raft_consensus.cc:348] T 60e8272de78a4c94b77408c4d1d5f60e P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:26.875773 4963 raft_consensus.cc:374] T 60e8272de78a4c94b77408c4d1d5f60e P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:26.875805 4963 raft_consensus.cc:729] T 60e8272de78a4c94b77408c4d1d5f60e P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:26.875921 4963 consensus_queue.cc:260] T 60e8272de78a4c94b77408c4d1d5f60e P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:26.876209 4963 ts_tablet_manager.cc:1398] T 60e8272de78a4c94b77408c4d1d5f60e P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:26.876722 4450 tablet_bootstrap.cc:492] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:26.876787 4450 ts_tablet_manager.cc:1367] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.001s
I20240502 20:46:26.877184 4450 raft_consensus.cc:348] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:26.877281 4450 raft_consensus.cc:374] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:26.877312 4450 raft_consensus.cc:729] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:26.877409 4450 consensus_queue.cc:260] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:26.877650 4450 ts_tablet_manager.cc:1398] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:26.877789 4450 tablet_bootstrap.cc:492] T 60e8272de78a4c94b77408c4d1d5f60e P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:26.878082 4965 tablet_bootstrap.cc:492] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:26.878216 4965 ts_tablet_manager.cc:1367] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 20:46:26.878593 4450 tablet_bootstrap.cc:654] T 60e8272de78a4c94b77408c4d1d5f60e P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:26.878870 4965 raft_consensus.cc:348] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:26.878965 4965 raft_consensus.cc:374] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:26.878996 4965 raft_consensus.cc:729] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:26.879103 4965 consensus_queue.cc:260] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:26.879350 4965 ts_tablet_manager.cc:1398] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:26.879493 4965 tablet_bootstrap.cc:492] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:26.885475 4965 tablet_bootstrap.cc:654] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:26.886875 4965 tablet_bootstrap.cc:492] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:26.886996 4965 ts_tablet_manager.cc:1367] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 20:46:26.887418 4965 raft_consensus.cc:348] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:26.887569 4965 raft_consensus.cc:374] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:26.887637 4965 raft_consensus.cc:729] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:26.887792 4965 consensus_queue.cc:260] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:26.888092 4965 ts_tablet_manager.cc:1398] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:26.899102 4450 tablet_bootstrap.cc:492] T 60e8272de78a4c94b77408c4d1d5f60e P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:26.899187 4450 ts_tablet_manager.cc:1367] T 60e8272de78a4c94b77408c4d1d5f60e P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.001s
I20240502 20:46:26.899611 4450 raft_consensus.cc:348] T 60e8272de78a4c94b77408c4d1d5f60e P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:26.899721 4450 raft_consensus.cc:374] T 60e8272de78a4c94b77408c4d1d5f60e P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:26.899750 4450 raft_consensus.cc:729] T 60e8272de78a4c94b77408c4d1d5f60e P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:26.899801 4961 tablet_bootstrap.cc:492] T 361b41488c5347e0aef692a39cd347c4 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:26.899878 4961 ts_tablet_manager.cc:1367] T 361b41488c5347e0aef692a39cd347c4 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20240502 20:46:26.899855 4450 consensus_queue.cc:260] T 60e8272de78a4c94b77408c4d1d5f60e P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:26.900120 4450 ts_tablet_manager.cc:1398] T 60e8272de78a4c94b77408c4d1d5f60e P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:26.901376 4961 raft_consensus.cc:348] T 361b41488c5347e0aef692a39cd347c4 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:26.901479 4961 raft_consensus.cc:374] T 361b41488c5347e0aef692a39cd347c4 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:26.901520 4961 raft_consensus.cc:729] T 361b41488c5347e0aef692a39cd347c4 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:26.901619 4961 consensus_queue.cc:260] T 361b41488c5347e0aef692a39cd347c4 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:26.901836 4961 ts_tablet_manager.cc:1398] T 361b41488c5347e0aef692a39cd347c4 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:26.904372 29175 consensus_queue.cc:237] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:26.905664 31624 raft_consensus.cc:1261] T 4a91eafc5ecf45b09f60d712884b7c79 P 315915efcf2b413ca7f9d874338e1b2d [term 2 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 20:46:26.905699 29315 raft_consensus.cc:1261] T 4a91eafc5ecf45b09f60d712884b7c79 P 09258ac095664fa99caf0707262cde13 [term 2 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 20:46:26.906011 3642 consensus_queue.cc:1035] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 20:46:26.906351 3642 consensus_queue.cc:1035] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 20:46:26.909071 4122 raft_consensus.cc:2932] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:26.909512 29315 raft_consensus.cc:2932] T 4a91eafc5ecf45b09f60d712884b7c79 P 09258ac095664fa99caf0707262cde13 [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:26.911830 29036 catalog_manager.cc:5484] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d reported cstate change: config changed from index 4 to 6, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New cstate: current_term: 2 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:26.914180 29425 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a91eafc5ecf45b09f60d712884b7c79 with delete_type TABLET_DATA_TOMBSTONED (TS dbcdc6ac635b44b185c3eba63f66e9d6 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:26.914799 4990 tablet_replica.cc:330] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:26.915063 4990 raft_consensus.cc:2229] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:26.915246 4990 raft_consensus.cc:2258] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:26.915697 4990 ts_tablet_manager.cc:1875] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:26.917287 31624 raft_consensus.cc:2932] T 4a91eafc5ecf45b09f60d712884b7c79 P 315915efcf2b413ca7f9d874338e1b2d [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:26.917919 4990 ts_tablet_manager.cc:1888] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 20:46:26.918031 4990 log.cc:1196] T 4a91eafc5ecf45b09f60d712884b7c79 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/4a91eafc5ecf45b09f60d712884b7c79
I20240502 20:46:26.918627 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet 4a91eafc5ecf45b09f60d712884b7c79 (table rebalance_test_table_7 [id=cf326efc9cf1428897a47dd98c65f088]) successfully deleted
I20240502 20:46:26.936295 4729 raft_consensus.cc:482] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:26.936393 4729 raft_consensus.cc:504] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:26.936767 4729 leader_election.cc:288] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:26.937177 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27bb5cc8379f44f49a9f7d1cc6ae1311" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:26.937228 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27bb5cc8379f44f49a9f7d1cc6ae1311" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:26.937299 29446 raft_consensus.cc:2454] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:26.937328 31888 raft_consensus.cc:2454] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:26.937683 29250 leader_election.cc:302] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:26.937912 4729 raft_consensus.cc:2781] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:26.937968 4729 raft_consensus.cc:482] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:26.938007 4729 raft_consensus.cc:3037] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:26.939009 4729 raft_consensus.cc:504] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:26.939275 4729 leader_election.cc:288] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:26.939602 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27bb5cc8379f44f49a9f7d1cc6ae1311" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:26.939641 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27bb5cc8379f44f49a9f7d1cc6ae1311" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:26.939710 29446 raft_consensus.cc:3037] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:26.939745 31888 raft_consensus.cc:3037] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:26.940506 29446 raft_consensus.cc:2454] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:26.940544 31888 raft_consensus.cc:2454] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:26.940821 29249 leader_election.cc:302] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:26.940984 4729 raft_consensus.cc:2781] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:26.941064 4729 raft_consensus.cc:686] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:26.941195 4729 consensus_queue.cc:237] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:26.943305 29033 catalog_manager.cc:5484] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:26.956842 29175 consensus_queue.cc:237] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:26.958122 29315 raft_consensus.cc:1261] T 035762cba9014632ad520f4777544641 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 20:46:26.958148 29117 consensus_peers.cc:485] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d -> Peer e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): Couldn't send request to peer e7ab6b9053664ee5aa71c7978e26fe6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 035762cba9014632ad520f4777544641. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:26.958494 29446 raft_consensus.cc:1261] T 035762cba9014632ad520f4777544641 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:26.958484 3642 consensus_queue.cc:1035] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:26.958804 3642 consensus_queue.cc:1035] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:26.960386 4122 raft_consensus.cc:2932] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:26.960773 29446 raft_consensus.cc:2932] T 035762cba9014632ad520f4777544641 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:26.962643 29315 raft_consensus.cc:2932] T 035762cba9014632ad520f4777544641 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:26.963301 29033 catalog_manager.cc:5484] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d reported cstate change: config changed from index -1 to 4, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:26.967383 4216 raft_consensus.cc:482] T 361b41488c5347e0aef692a39cd347c4 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:26.967478 4216 raft_consensus.cc:504] T 361b41488c5347e0aef692a39cd347c4 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:26.967844 4216 leader_election.cc:288] T 361b41488c5347e0aef692a39cd347c4 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:26.968254 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "361b41488c5347e0aef692a39cd347c4" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:26.968389 31755 raft_consensus.cc:2454] T 361b41488c5347e0aef692a39cd347c4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:26.968623 29510 leader_election.cc:302] T 361b41488c5347e0aef692a39cd347c4 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:26.968796 4216 raft_consensus.cc:2781] T 361b41488c5347e0aef692a39cd347c4 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:26.968838 4216 raft_consensus.cc:482] T 361b41488c5347e0aef692a39cd347c4 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:26.968870 4216 raft_consensus.cc:3037] T 361b41488c5347e0aef692a39cd347c4 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:26.969105 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "361b41488c5347e0aef692a39cd347c4" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:26.969209 29446 raft_consensus.cc:2454] T 361b41488c5347e0aef692a39cd347c4 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:26.969831 4216 raft_consensus.cc:504] T 361b41488c5347e0aef692a39cd347c4 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:26.970152 4216 leader_election.cc:288] T 361b41488c5347e0aef692a39cd347c4 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Requested vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:26.970417 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "361b41488c5347e0aef692a39cd347c4" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:26.970538 29446 raft_consensus.cc:3037] T 361b41488c5347e0aef692a39cd347c4 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:26.970585 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "361b41488c5347e0aef692a39cd347c4" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:26.970688 31755 raft_consensus.cc:3037] T 361b41488c5347e0aef692a39cd347c4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:26.971354 29446 raft_consensus.cc:2454] T 361b41488c5347e0aef692a39cd347c4 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:26.971621 29512 leader_election.cc:302] T 361b41488c5347e0aef692a39cd347c4 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:26.971648 31755 raft_consensus.cc:2454] T 361b41488c5347e0aef692a39cd347c4 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:26.971766 4216 raft_consensus.cc:2781] T 361b41488c5347e0aef692a39cd347c4 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:26.971845 4216 raft_consensus.cc:686] T 361b41488c5347e0aef692a39cd347c4 P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Becoming Leader. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Running, Role: LEADER
I20240502 20:46:26.971966 4216 consensus_queue.cc:237] T 361b41488c5347e0aef692a39cd347c4 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:26.974531 29036 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:26.974478 29042 catalog_manager.cc:5484] T 361b41488c5347e0aef692a39cd347c4 P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: term changed from 0 to 1, leader changed from <none> to db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:26.980747 4678 tablet.cc:1722] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:26.981133 4944 tablet.cc:1722] T 8db888c800cf41ddb08b3d38180aacef P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:26.983270 4438 tablet.cc:1722] T 8db888c800cf41ddb08b3d38180aacef P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:26.985239 4694 raft_consensus.cc:482] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:26.985335 4694 raft_consensus.cc:504] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:26.985670 4694 leader_election.cc:288] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:26.985945 29036 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=c16a44861d9a4e0f9abf52b84940f2c7] alter complete (version 3)
I20240502 20:46:26.986011 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60e8272de78a4c94b77408c4d1d5f60e" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:26.986140 31494 raft_consensus.cc:2454] T 60e8272de78a4c94b77408c4d1d5f60e P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:26.986423 29381 leader_election.cc:302] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:26.986675 4694 raft_consensus.cc:2781] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:26.986728 4694 raft_consensus.cc:482] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:26.986759 4694 raft_consensus.cc:3037] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:26.986747 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60e8272de78a4c94b77408c4d1d5f60e" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:26.986836 31888 raft_consensus.cc:2454] T 60e8272de78a4c94b77408c4d1d5f60e P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:26.987648 4694 raft_consensus.cc:504] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:26.987964 4694 leader_election.cc:288] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:26.988209 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60e8272de78a4c94b77408c4d1d5f60e" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:26.988251 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60e8272de78a4c94b77408c4d1d5f60e" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:26.988317 31888 raft_consensus.cc:3037] T 60e8272de78a4c94b77408c4d1d5f60e P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:26.988356 31494 raft_consensus.cc:3037] T 60e8272de78a4c94b77408c4d1d5f60e P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:26.989249 31494 raft_consensus.cc:2454] T 60e8272de78a4c94b77408c4d1d5f60e P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:26.989287 31888 raft_consensus.cc:2454] T 60e8272de78a4c94b77408c4d1d5f60e P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:26.989504 29381 leader_election.cc:302] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:26.989710 4694 raft_consensus.cc:2781] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:26.989784 4694 raft_consensus.cc:686] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:26.989904 4694 consensus_queue.cc:237] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:26.991726 29033 catalog_manager.cc:5484] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:26.997104 29175 consensus_queue.cc:237] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:26.998348 29446 raft_consensus.cc:1261] T 849c31557dec4043ad36dbddc6811b3f P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:26.998349 29577 raft_consensus.cc:1261] T 849c31557dec4043ad36dbddc6811b3f P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 20:46:26.998497 29119 consensus_peers.cc:485] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d -> Peer 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): Couldn't send request to peer 2a9c1af8cf814030bde56b67ca5c2fbe. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 849c31557dec4043ad36dbddc6811b3f. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:26.998670 3642 consensus_queue.cc:1035] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:26.998843 5032 consensus_queue.cc:1035] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:27.000727 3642 raft_consensus.cc:2932] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:27.001184 29577 raft_consensus.cc:2932] T 849c31557dec4043ad36dbddc6811b3f P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:27.003988 29033 catalog_manager.cc:5484] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d reported cstate change: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:27.005324 29446 raft_consensus.cc:2932] T 849c31557dec4043ad36dbddc6811b3f P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:27.028786 29175 consensus_queue.cc:237] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:27.030208 29315 raft_consensus.cc:1261] T b28fe6bccf4f48a6a5456e6ead5cc920 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:27.030206 29577 raft_consensus.cc:1261] T b28fe6bccf4f48a6a5456e6ead5cc920 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 20:46:27.030431 29118 consensus_peers.cc:485] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b28fe6bccf4f48a6a5456e6ead5cc920. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:27.030596 5032 consensus_queue.cc:1035] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:27.030753 5033 consensus_queue.cc:1035] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:27.034935 5032 raft_consensus.cc:2932] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:27.035547 29577 raft_consensus.cc:2932] T b28fe6bccf4f48a6a5456e6ead5cc920 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:27.037137 29315 raft_consensus.cc:2932] T b28fe6bccf4f48a6a5456e6ead5cc920 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:27.037820 29042 catalog_manager.cc:5484] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d reported cstate change: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:27.088274 29039 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:27.088392 29039 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:27.091269 29039 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 8db888c800cf41ddb08b3d38180aacef
I20240502 20:46:27.092028 29163 tablet_service.cc:1514] Processing DeleteTablet for tablet 8db888c800cf41ddb08b3d38180aacef with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:27 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:27.092334 31605 tablet_service.cc:1514] Processing DeleteTablet for tablet 8db888c800cf41ddb08b3d38180aacef with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:27 UTC) from {username='slave'} at 127.0.0.1:58346
I20240502 20:46:27.092372 31867 tablet_service.cc:1514] Processing DeleteTablet for tablet 8db888c800cf41ddb08b3d38180aacef with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:27 UTC) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:27.092542 4773 tablet_replica.cc:330] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:27.092682 4773 raft_consensus.cc:2229] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:27.092877 4773 raft_consensus.cc:2258] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:27.093119 5115 tablet_replica.cc:330] T 8db888c800cf41ddb08b3d38180aacef P 315915efcf2b413ca7f9d874338e1b2d: stopping tablet replica
I20240502 20:46:27.093350 5115 raft_consensus.cc:2229] T 8db888c800cf41ddb08b3d38180aacef P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:27.093441 4773 ts_tablet_manager.cc:1875] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:27.093542 5115 raft_consensus.cc:2258] T 8db888c800cf41ddb08b3d38180aacef P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:27.094123 5115 ts_tablet_manager.cc:1875] T 8db888c800cf41ddb08b3d38180aacef P 315915efcf2b413ca7f9d874338e1b2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:27.095927 4773 ts_tablet_manager.cc:1888] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:27.096015 4773 log.cc:1196] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/8db888c800cf41ddb08b3d38180aacef
I20240502 20:46:27.096236 5115 ts_tablet_manager.cc:1888] T 8db888c800cf41ddb08b3d38180aacef P 315915efcf2b413ca7f9d874338e1b2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:27.096309 5115 log.cc:1196] T 8db888c800cf41ddb08b3d38180aacef P 315915efcf2b413ca7f9d874338e1b2d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal/wals/8db888c800cf41ddb08b3d38180aacef
I20240502 20:46:27.096387 4773 ts_tablet_manager.cc:1909] T 8db888c800cf41ddb08b3d38180aacef P e6b60803d0764d228c863e2da65e3691: Deleting consensus metadata
I20240502 20:46:27.096588 5115 ts_tablet_manager.cc:1909] T 8db888c800cf41ddb08b3d38180aacef P 315915efcf2b413ca7f9d874338e1b2d: Deleting consensus metadata
I20240502 20:46:27.097172 29027 catalog_manager.cc:4830] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): tablet 8db888c800cf41ddb08b3d38180aacef (table rebalancer_dynamic_table_new_name [id=c16a44861d9a4e0f9abf52b84940f2c7]) successfully deleted
I20240502 20:46:27.097318 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet 8db888c800cf41ddb08b3d38180aacef (table rebalancer_dynamic_table_new_name [id=c16a44861d9a4e0f9abf52b84940f2c7]) successfully deleted
I20240502 20:46:27.099910 5114 tablet_replica.cc:330] T 8db888c800cf41ddb08b3d38180aacef P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:27.100101 5114 raft_consensus.cc:2229] T 8db888c800cf41ddb08b3d38180aacef P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:27.100317 5114 raft_consensus.cc:2258] T 8db888c800cf41ddb08b3d38180aacef P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:27.101250 5114 ts_tablet_manager.cc:1875] T 8db888c800cf41ddb08b3d38180aacef P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:27.110288 5114 ts_tablet_manager.cc:1888] T 8db888c800cf41ddb08b3d38180aacef P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:27.110407 5114 log.cc:1196] T 8db888c800cf41ddb08b3d38180aacef P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/8db888c800cf41ddb08b3d38180aacef
I20240502 20:46:27.111271 5114 ts_tablet_manager.cc:1909] T 8db888c800cf41ddb08b3d38180aacef P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:27.111927 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 8db888c800cf41ddb08b3d38180aacef (table rebalancer_dynamic_table_new_name [id=c16a44861d9a4e0f9abf52b84940f2c7]) successfully deleted
I20240502 20:46:27.129528 31605 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:27.132776 29295 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:27.138026 31735 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:27.139863 29163 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:27.143738 29425 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:27.147876 31867 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:27.151769 31475 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:27.165561 29556 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:27.193564 29038 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:27.198942 31735 tablet_service.cc:1467] Processing CreateTablet for tablet aa95224f07674eb28983ee58f319d168 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=787c6706bb3c4474950f26a0f0ee080d]), partition=
I20240502 20:46:27.199168 31735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa95224f07674eb28983ee58f319d168. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:27.201198 4957 tablet_bootstrap.cc:492] T aa95224f07674eb28983ee58f319d168 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:27.201978 29163 tablet_service.cc:1467] Processing CreateTablet for tablet aa95224f07674eb28983ee58f319d168 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=787c6706bb3c4474950f26a0f0ee080d]), partition=
I20240502 20:46:27.202172 4957 tablet_bootstrap.cc:654] T aa95224f07674eb28983ee58f319d168 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:27.202224 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa95224f07674eb28983ee58f319d168. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:27.203027 31867 tablet_service.cc:1467] Processing CreateTablet for tablet aa95224f07674eb28983ee58f319d168 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=787c6706bb3c4474950f26a0f0ee080d]), partition=
I20240502 20:46:27.203239 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa95224f07674eb28983ee58f319d168. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:27.203295 4957 tablet_bootstrap.cc:492] T aa95224f07674eb28983ee58f319d168 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:27.203373 4957 ts_tablet_manager.cc:1367] T aa95224f07674eb28983ee58f319d168 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:27.203783 4957 raft_consensus.cc:348] T aa95224f07674eb28983ee58f319d168 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:27.203877 4957 raft_consensus.cc:374] T aa95224f07674eb28983ee58f319d168 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:27.203928 4957 raft_consensus.cc:729] T aa95224f07674eb28983ee58f319d168 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:27.204048 4957 consensus_queue.cc:260] T aa95224f07674eb28983ee58f319d168 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:27.204339 4957 ts_tablet_manager.cc:1398] T aa95224f07674eb28983ee58f319d168 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:27.204710 5128 tablet_bootstrap.cc:492] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:27.205057 4450 tablet_bootstrap.cc:492] T aa95224f07674eb28983ee58f319d168 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:27.206028 4450 tablet_bootstrap.cc:654] T aa95224f07674eb28983ee58f319d168 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:27.206126 5128 tablet_bootstrap.cc:654] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:27.207346 5128 tablet_bootstrap.cc:492] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:27.207739 5128 ts_tablet_manager.cc:1367] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 20:46:27.208189 5128 raft_consensus.cc:348] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:27.208343 5128 raft_consensus.cc:374] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:27.208410 5128 raft_consensus.cc:729] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:27.208570 5128 consensus_queue.cc:260] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:27.208835 5128 ts_tablet_manager.cc:1398] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:27.215420 4450 tablet_bootstrap.cc:492] T aa95224f07674eb28983ee58f319d168 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:27.215505 4450 ts_tablet_manager.cc:1367] T aa95224f07674eb28983ee58f319d168 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 20:46:27.216107 4450 raft_consensus.cc:348] T aa95224f07674eb28983ee58f319d168 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:27.216219 4450 raft_consensus.cc:374] T aa95224f07674eb28983ee58f319d168 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:27.216249 4450 raft_consensus.cc:729] T aa95224f07674eb28983ee58f319d168 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:27.216367 4450 consensus_queue.cc:260] T aa95224f07674eb28983ee58f319d168 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:27.216614 4450 ts_tablet_manager.cc:1398] T aa95224f07674eb28983ee58f319d168 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:27.301529 5033 raft_consensus.cc:482] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:27.301648 5033 raft_consensus.cc:504] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:27.302102 5033 leader_election.cc:288] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:27.302379 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa95224f07674eb28983ee58f319d168" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:27.302541 31755 raft_consensus.cc:2454] T aa95224f07674eb28983ee58f319d168 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:27.302784 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa95224f07674eb28983ee58f319d168" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:27.302899 31888 raft_consensus.cc:2454] T aa95224f07674eb28983ee58f319d168 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:27.303133 29118 leader_election.cc:302] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:27.303385 5033 raft_consensus.cc:2781] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:27.303440 5033 raft_consensus.cc:482] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:27.303483 5033 raft_consensus.cc:3037] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:27.304440 5033 raft_consensus.cc:504] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:27.304750 5033 leader_election.cc:288] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:27.305054 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa95224f07674eb28983ee58f319d168" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:27.305166 31755 raft_consensus.cc:3037] T aa95224f07674eb28983ee58f319d168 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:27.306008 31755 raft_consensus.cc:2454] T aa95224f07674eb28983ee58f319d168 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:27.306231 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa95224f07674eb28983ee58f319d168" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:27.306288 29117 leader_election.cc:302] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:27.306327 31888 raft_consensus.cc:3037] T aa95224f07674eb28983ee58f319d168 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:27.306422 5033 raft_consensus.cc:2781] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:27.306531 5033 raft_consensus.cc:686] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:27.306682 5033 consensus_queue.cc:237] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:27.307268 31888 raft_consensus.cc:2454] T aa95224f07674eb28983ee58f319d168 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:27.308691 29033 catalog_manager.cc:5484] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:27.330623 29163 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:27.332366 29295 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:27.340658 29556 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:27.342108 31867 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:27.342913 31605 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:27.348349 29425 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:27.355803 31475 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:27.358696 31735 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:27.364104 4729 consensus_queue.cc:1035] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:27.372782 4444 consensus_queue.cc:1035] T 27bb5cc8379f44f49a9f7d1cc6ae1311 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:27.386760 5188 ts_tablet_manager.cc:902] T 849c31557dec4043ad36dbddc6811b3f P 2a9c1af8cf814030bde56b67ca5c2fbe: Initiating tablet copy from peer 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:27.394838 5188 tablet_copy_client.cc:308] T 849c31557dec4043ad36dbddc6811b3f P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.65:43267
I20240502 20:46:27.397015 29202 tablet_copy_service.cc:140] P 7796ea3fba294d17a34386873969431d: Received BeginTabletCopySession request for tablet 849c31557dec4043ad36dbddc6811b3f from peer 2a9c1af8cf814030bde56b67ca5c2fbe ({username='slave'} at 127.22.45.69:49017)
I20240502 20:46:27.397119 29202 tablet_copy_service.cc:161] P 7796ea3fba294d17a34386873969431d: Beginning new tablet copy session on tablet 849c31557dec4043ad36dbddc6811b3f from peer 2a9c1af8cf814030bde56b67ca5c2fbe at {username='slave'} at 127.22.45.69:49017: session id = 2a9c1af8cf814030bde56b67ca5c2fbe-849c31557dec4043ad36dbddc6811b3f
I20240502 20:46:27.397814 29202 tablet_copy_source_session.cc:205] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:27.404067 5188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 849c31557dec4043ad36dbddc6811b3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:27.404364 4216 consensus_queue.cc:1035] T 361b41488c5347e0aef692a39cd347c4 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:27.406360 5188 tablet_copy_client.cc:783] T 849c31557dec4043ad36dbddc6811b3f P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:27.408033 5188 tablet_copy_client.cc:647] T 849c31557dec4043ad36dbddc6811b3f P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:27.409399 5188 tablet_copy_client.cc:523] T 849c31557dec4043ad36dbddc6811b3f P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:27.410108 4216 consensus_queue.cc:1035] T 361b41488c5347e0aef692a39cd347c4 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:27.412132 5188 tablet_bootstrap.cc:492] T 849c31557dec4043ad36dbddc6811b3f P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:27.414894 4694 consensus_queue.cc:1035] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:27.421577 4694 consensus_queue.cc:1035] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:27.427419 29039 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:27.431556 31888 raft_consensus.cc:1261] T aa95224f07674eb28983ee58f319d168 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:27.431881 5032 consensus_queue.cc:1035] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:27.431913 31755 raft_consensus.cc:1261] T aa95224f07674eb28983ee58f319d168 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:27.432394 5032 consensus_queue.cc:1035] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:27.436282 29033 catalog_manager.cc:5576] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787) does not have the latest schema for tablet aa95224f07674eb28983ee58f319d168 (table rebalancer_dynamic_table [id=787c6706bb3c4474950f26a0f0ee080d]). Expected version 1 got 0
I20240502 20:46:27.436446 4944 tablet.cc:1722] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:27.436755 4944 tablet.cc:1620] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d: MemRowSet was empty: no flush needed.
I20240502 20:46:27.437093 29039 catalog_manager.cc:5576] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207) does not have the latest schema for tablet aa95224f07674eb28983ee58f319d168 (table rebalancer_dynamic_table [id=787c6706bb3c4474950f26a0f0ee080d]). Expected version 1 got 0
I20240502 20:46:27.437314 4678 tablet.cc:1722] T aa95224f07674eb28983ee58f319d168 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:27.437575 4678 tablet.cc:1620] T aa95224f07674eb28983ee58f319d168 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:27.439607 29035 catalog_manager.cc:5576] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787) does not have the latest schema for tablet aa95224f07674eb28983ee58f319d168 (table rebalancer_dynamic_table [id=787c6706bb3c4474950f26a0f0ee080d]). Expected version 1 got 0
I20240502 20:46:27.441434 5214 tablet.cc:1722] T aa95224f07674eb28983ee58f319d168 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:27.441711 5214 tablet.cc:1620] T aa95224f07674eb28983ee58f319d168 P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:27.442080 29033 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=787c6706bb3c4474950f26a0f0ee080d] alter complete (version 1)
I20240502 20:46:27.452338 5188 tablet_bootstrap.cc:492] T 849c31557dec4043ad36dbddc6811b3f P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:27.452610 5188 tablet_bootstrap.cc:492] T 849c31557dec4043ad36dbddc6811b3f P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap complete.
I20240502 20:46:27.452797 5188 ts_tablet_manager.cc:1367] T 849c31557dec4043ad36dbddc6811b3f P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.041s user 0.028s sys 0.000s
I20240502 20:46:27.453219 5188 raft_consensus.cc:348] T 849c31557dec4043ad36dbddc6811b3f P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:27.453414 5188 raft_consensus.cc:729] T 849c31557dec4043ad36dbddc6811b3f P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:27.453589 5188 consensus_queue.cc:260] T 849c31557dec4043ad36dbddc6811b3f P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:27.453896 5188 ts_tablet_manager.cc:1398] T 849c31557dec4043ad36dbddc6811b3f P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 20:46:27.454357 29202 tablet_copy_service.cc:342] P 7796ea3fba294d17a34386873969431d: Request end of tablet copy session 2a9c1af8cf814030bde56b67ca5c2fbe-849c31557dec4043ad36dbddc6811b3f received from {username='slave'} at 127.22.45.69:49017
I20240502 20:46:27.454430 29202 tablet_copy_service.cc:433] P 7796ea3fba294d17a34386873969431d: ending tablet copy session 2a9c1af8cf814030bde56b67ca5c2fbe-849c31557dec4043ad36dbddc6811b3f on tablet 849c31557dec4043ad36dbddc6811b3f with peer 2a9c1af8cf814030bde56b67ca5c2fbe
I20240502 20:46:27.479617 5226 ts_tablet_manager.cc:902] T 035762cba9014632ad520f4777544641 P e7ab6b9053664ee5aa71c7978e26fe6a: Initiating tablet copy from peer 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:27.480360 5226 tablet_copy_client.cc:308] T 035762cba9014632ad520f4777544641 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.65:43267
I20240502 20:46:27.480809 29202 tablet_copy_service.cc:140] P 7796ea3fba294d17a34386873969431d: Received BeginTabletCopySession request for tablet 035762cba9014632ad520f4777544641 from peer e7ab6b9053664ee5aa71c7978e26fe6a ({username='slave'} at 127.22.45.71:50917)
I20240502 20:46:27.480912 29202 tablet_copy_service.cc:161] P 7796ea3fba294d17a34386873969431d: Beginning new tablet copy session on tablet 035762cba9014632ad520f4777544641 from peer e7ab6b9053664ee5aa71c7978e26fe6a at {username='slave'} at 127.22.45.71:50917: session id = e7ab6b9053664ee5aa71c7978e26fe6a-035762cba9014632ad520f4777544641
I20240502 20:46:27.481860 29202 tablet_copy_source_session.cc:205] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:27.484463 5226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 035762cba9014632ad520f4777544641. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:27.487349 5226 tablet_copy_client.cc:783] T 035762cba9014632ad520f4777544641 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:27.487504 5226 tablet_copy_client.cc:647] T 035762cba9014632ad520f4777544641 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:27.489081 5226 tablet_copy_client.cc:523] T 035762cba9014632ad520f4777544641 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:27.490793 5226 tablet_bootstrap.cc:492] T 035762cba9014632ad520f4777544641 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:27.517855 5226 tablet_bootstrap.cc:492] T 035762cba9014632ad520f4777544641 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:27.518141 5226 tablet_bootstrap.cc:492] T 035762cba9014632ad520f4777544641 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap complete.
I20240502 20:46:27.518321 5226 ts_tablet_manager.cc:1367] T 035762cba9014632ad520f4777544641 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.028s user 0.019s sys 0.004s
I20240502 20:46:27.518764 5226 raft_consensus.cc:348] T 035762cba9014632ad520f4777544641 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:27.518877 5226 raft_consensus.cc:729] T 035762cba9014632ad520f4777544641 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:27.519265 5226 consensus_queue.cc:260] T 035762cba9014632ad520f4777544641 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:27.519349 29033 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_40"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:27.519565 5226 ts_tablet_manager.cc:1398] T 035762cba9014632ad520f4777544641 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 20:46:27.521063 29202 tablet_copy_service.cc:342] P 7796ea3fba294d17a34386873969431d: Request end of tablet copy session e7ab6b9053664ee5aa71c7978e26fe6a-035762cba9014632ad520f4777544641 received from {username='slave'} at 127.22.45.71:50917
I20240502 20:46:27.521138 29202 tablet_copy_service.cc:433] P 7796ea3fba294d17a34386873969431d: ending tablet copy session e7ab6b9053664ee5aa71c7978e26fe6a-035762cba9014632ad520f4777544641 on tablet 035762cba9014632ad520f4777544641 with peer e7ab6b9053664ee5aa71c7978e26fe6a
I20240502 20:46:27.533272 29556 tablet_service.cc:1467] Processing CreateTablet for tablet 9992a7ea196645dc82d2430fcb405254 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=833f1c6e07ac4f5fa5be5509611305c3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:27.533581 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9992a7ea196645dc82d2430fcb405254. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:27.536331 31605 tablet_service.cc:1467] Processing CreateTablet for tablet 9992a7ea196645dc82d2430fcb405254 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=833f1c6e07ac4f5fa5be5509611305c3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:27.536585 31605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9992a7ea196645dc82d2430fcb405254. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:27.536628 29163 tablet_service.cc:1467] Processing CreateTablet for tablet 370bf1bcfb3343c488954ece2488b4c2 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=833f1c6e07ac4f5fa5be5509611305c3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:27.536747 29293 tablet_service.cc:1467] Processing CreateTablet for tablet 370bf1bcfb3343c488954ece2488b4c2 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=833f1c6e07ac4f5fa5be5509611305c3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:27.536880 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 370bf1bcfb3343c488954ece2488b4c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:27.537019 29293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 370bf1bcfb3343c488954ece2488b4c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:27.539124 5255 tablet_bootstrap.cc:492] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:27.539227 29294 tablet_service.cc:1467] Processing CreateTablet for tablet 4d040ba420e246b6b1f6a736bde65803 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=833f1c6e07ac4f5fa5be5509611305c3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:27.539435 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d040ba420e246b6b1f6a736bde65803. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:27.541335 5255 tablet_bootstrap.cc:654] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:27.541481 5128 tablet_bootstrap.cc:492] T 370bf1bcfb3343c488954ece2488b4c2 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:27.542433 5128 tablet_bootstrap.cc:654] T 370bf1bcfb3343c488954ece2488b4c2 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:27.543684 31475 tablet_service.cc:1467] Processing CreateTablet for tablet 4d040ba420e246b6b1f6a736bde65803 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=833f1c6e07ac4f5fa5be5509611305c3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:27.536717 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 9992a7ea196645dc82d2430fcb405254 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=833f1c6e07ac4f5fa5be5509611305c3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:27.543970 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d040ba420e246b6b1f6a736bde65803. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:27.544046 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9992a7ea196645dc82d2430fcb405254. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:27.546258 5253 tablet_bootstrap.cc:492] T 9992a7ea196645dc82d2430fcb405254 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:27.547214 5253 tablet_bootstrap.cc:654] T 9992a7ea196645dc82d2430fcb405254 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:27.547690 29036 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:27.548036 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 4d040ba420e246b6b1f6a736bde65803 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=833f1c6e07ac4f5fa5be5509611305c3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:27.548333 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d040ba420e246b6b1f6a736bde65803. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:27.548224 29556 tablet_service.cc:1467] Processing CreateTablet for tablet 370bf1bcfb3343c488954ece2488b4c2 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=833f1c6e07ac4f5fa5be5509611305c3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:27.549333 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 370bf1bcfb3343c488954ece2488b4c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:27.549670 5128 tablet_bootstrap.cc:492] T 370bf1bcfb3343c488954ece2488b4c2 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:27.549757 5128 ts_tablet_manager.cc:1367] T 370bf1bcfb3343c488954ece2488b4c2 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 20:46:27.550277 5128 raft_consensus.cc:348] T 370bf1bcfb3343c488954ece2488b4c2 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:27.550387 5128 raft_consensus.cc:374] T 370bf1bcfb3343c488954ece2488b4c2 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:27.550421 5128 raft_consensus.cc:729] T 370bf1bcfb3343c488954ece2488b4c2 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:27.550604 4450 tablet_bootstrap.cc:492] T 4d040ba420e246b6b1f6a736bde65803 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:27.550566 5128 consensus_queue.cc:260] T 370bf1bcfb3343c488954ece2488b4c2 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:27.550902 5128 ts_tablet_manager.cc:1398] T 370bf1bcfb3343c488954ece2488b4c2 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:27.551620 4450 tablet_bootstrap.cc:654] T 4d040ba420e246b6b1f6a736bde65803 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:27.553687 5253 tablet_bootstrap.cc:492] T 9992a7ea196645dc82d2430fcb405254 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:27.553812 5253 ts_tablet_manager.cc:1367] T 9992a7ea196645dc82d2430fcb405254 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 20:46:27.554340 5253 raft_consensus.cc:348] T 9992a7ea196645dc82d2430fcb405254 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:27.554507 5253 raft_consensus.cc:374] T 9992a7ea196645dc82d2430fcb405254 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:27.554569 5253 raft_consensus.cc:729] T 9992a7ea196645dc82d2430fcb405254 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:27.554783 5253 consensus_queue.cc:260] T 9992a7ea196645dc82d2430fcb405254 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:27.555155 5253 ts_tablet_manager.cc:1398] T 9992a7ea196645dc82d2430fcb405254 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:27.555447 4944 tablet.cc:1722] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:27.555502 5253 tablet_bootstrap.cc:492] T 370bf1bcfb3343c488954ece2488b4c2 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:27.555563 5255 tablet_bootstrap.cc:492] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:27.555722 4944 tablet.cc:1620] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d: MemRowSet was empty: no flush needed.
I20240502 20:46:27.555642 5255 ts_tablet_manager.cc:1367] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 20:46:27.556386 5255 raft_consensus.cc:348] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:27.556512 5255 raft_consensus.cc:374] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:27.556552 5255 raft_consensus.cc:729] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:27.556695 5255 consensus_queue.cc:260] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:27.557008 5255 ts_tablet_manager.cc:1398] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:27.557210 5255 tablet_bootstrap.cc:492] T 4d040ba420e246b6b1f6a736bde65803 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:27.557343 5253 tablet_bootstrap.cc:654] T 370bf1bcfb3343c488954ece2488b4c2 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:27.558609 5253 tablet_bootstrap.cc:492] T 370bf1bcfb3343c488954ece2488b4c2 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:27.558696 5253 ts_tablet_manager.cc:1367] T 370bf1bcfb3343c488954ece2488b4c2 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:27.559140 5253 raft_consensus.cc:348] T 370bf1bcfb3343c488954ece2488b4c2 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:27.559252 5253 raft_consensus.cc:374] T 370bf1bcfb3343c488954ece2488b4c2 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:27.559284 4450 tablet_bootstrap.cc:492] T 4d040ba420e246b6b1f6a736bde65803 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:27.559288 5253 raft_consensus.cc:729] T 370bf1bcfb3343c488954ece2488b4c2 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:27.559357 4450 ts_tablet_manager.cc:1367] T 4d040ba420e246b6b1f6a736bde65803 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20240502 20:46:27.559406 5253 consensus_queue.cc:260] T 370bf1bcfb3343c488954ece2488b4c2 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:27.559690 5253 ts_tablet_manager.cc:1398] T 370bf1bcfb3343c488954ece2488b4c2 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:27.559820 4678 tablet.cc:1722] T aa95224f07674eb28983ee58f319d168 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:27.560106 4678 tablet.cc:1620] T aa95224f07674eb28983ee58f319d168 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:27.560235 5255 tablet_bootstrap.cc:654] T 4d040ba420e246b6b1f6a736bde65803 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:27.561007 5258 tablet_bootstrap.cc:492] T 4d040ba420e246b6b1f6a736bde65803 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:27.561379 5255 tablet_bootstrap.cc:492] T 4d040ba420e246b6b1f6a736bde65803 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:27.561519 5255 ts_tablet_manager.cc:1367] T 4d040ba420e246b6b1f6a736bde65803 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:27.561898 5255 raft_consensus.cc:348] T 4d040ba420e246b6b1f6a736bde65803 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:27.561987 5255 raft_consensus.cc:374] T 4d040ba420e246b6b1f6a736bde65803 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:27.562019 5255 raft_consensus.cc:729] T 4d040ba420e246b6b1f6a736bde65803 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:27.563755 5255 consensus_queue.cc:260] T 4d040ba420e246b6b1f6a736bde65803 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:27.564157 5255 ts_tablet_manager.cc:1398] T 4d040ba420e246b6b1f6a736bde65803 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:27.564378 5255 tablet_bootstrap.cc:492] T 9992a7ea196645dc82d2430fcb405254 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:27.565203 5255 tablet_bootstrap.cc:654] T 9992a7ea196645dc82d2430fcb405254 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:27.565356 5214 tablet.cc:1722] T aa95224f07674eb28983ee58f319d168 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:27.565620 5214 tablet.cc:1620] T aa95224f07674eb28983ee58f319d168 P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:27.565857 5258 tablet_bootstrap.cc:654] T 4d040ba420e246b6b1f6a736bde65803 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:27.567037 5258 tablet_bootstrap.cc:492] T 4d040ba420e246b6b1f6a736bde65803 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:27.567180 5258 ts_tablet_manager.cc:1367] T 4d040ba420e246b6b1f6a736bde65803 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 20:46:27.567685 5258 raft_consensus.cc:348] T 4d040ba420e246b6b1f6a736bde65803 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:27.568341 5258 raft_consensus.cc:374] T 4d040ba420e246b6b1f6a736bde65803 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:27.568578 5255 tablet_bootstrap.cc:492] T 9992a7ea196645dc82d2430fcb405254 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:27.568675 5258 raft_consensus.cc:729] T 4d040ba420e246b6b1f6a736bde65803 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:27.559834 4450 raft_consensus.cc:348] T 4d040ba420e246b6b1f6a736bde65803 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:27.568872 4450 raft_consensus.cc:374] T 4d040ba420e246b6b1f6a736bde65803 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:27.568902 4450 raft_consensus.cc:729] T 4d040ba420e246b6b1f6a736bde65803 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:27.568922 5255 ts_tablet_manager.cc:1367] T 9992a7ea196645dc82d2430fcb405254 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 20:46:27.569001 4450 consensus_queue.cc:260] T 4d040ba420e246b6b1f6a736bde65803 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:27.569248 4450 ts_tablet_manager.cc:1398] T 4d040ba420e246b6b1f6a736bde65803 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.010s user 0.000s sys 0.000s
I20240502 20:46:27.569295 5255 raft_consensus.cc:348] T 9992a7ea196645dc82d2430fcb405254 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:27.569389 5255 raft_consensus.cc:374] T 9992a7ea196645dc82d2430fcb405254 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:27.569568 5258 consensus_queue.cc:260] T 4d040ba420e246b6b1f6a736bde65803 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:27.573635 5254 tablet_bootstrap.cc:492] T 9992a7ea196645dc82d2430fcb405254 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:27.576963 29037 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=787c6706bb3c4474950f26a0f0ee080d] alter complete (version 2)
I20240502 20:46:27.578732 5258 ts_tablet_manager.cc:1398] T 4d040ba420e246b6b1f6a736bde65803 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.011s user 0.001s sys 0.000s
I20240502 20:46:27.578847 5255 raft_consensus.cc:729] T 9992a7ea196645dc82d2430fcb405254 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:27.579124 5255 consensus_queue.cc:260] T 9992a7ea196645dc82d2430fcb405254 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:27.579571 5255 ts_tablet_manager.cc:1398] T 9992a7ea196645dc82d2430fcb405254 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.011s user 0.001s sys 0.000s
I20240502 20:46:27.580622 5254 tablet_bootstrap.cc:654] T 9992a7ea196645dc82d2430fcb405254 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:27.582439 5254 tablet_bootstrap.cc:492] T 9992a7ea196645dc82d2430fcb405254 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:27.582525 5254 ts_tablet_manager.cc:1367] T 9992a7ea196645dc82d2430fcb405254 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 20:46:27.582963 5254 raft_consensus.cc:348] T 9992a7ea196645dc82d2430fcb405254 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:27.583119 5254 raft_consensus.cc:374] T 9992a7ea196645dc82d2430fcb405254 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:27.583213 5254 raft_consensus.cc:729] T 9992a7ea196645dc82d2430fcb405254 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:27.583590 5254 consensus_queue.cc:260] T 9992a7ea196645dc82d2430fcb405254 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:27.585047 5254 ts_tablet_manager.cc:1398] T 9992a7ea196645dc82d2430fcb405254 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:27.594130 4444 raft_consensus.cc:482] T 4d040ba420e246b6b1f6a736bde65803 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:27.594219 4444 raft_consensus.cc:504] T 4d040ba420e246b6b1f6a736bde65803 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:27.594732 4444 leader_election.cc:288] T 4d040ba420e246b6b1f6a736bde65803 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:27.595023 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d040ba420e246b6b1f6a736bde65803" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:27.595142 31495 raft_consensus.cc:2454] T 4d040ba420e246b6b1f6a736bde65803 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:27.595144 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d040ba420e246b6b1f6a736bde65803" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:27.595257 31888 raft_consensus.cc:2454] T 4d040ba420e246b6b1f6a736bde65803 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:27.595546 29250 leader_election.cc:302] T 4d040ba420e246b6b1f6a736bde65803 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 2a9c1af8cf814030bde56b67ca5c2fbe; no voters:
I20240502 20:46:27.595757 4444 raft_consensus.cc:2781] T 4d040ba420e246b6b1f6a736bde65803 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:27.595806 4444 raft_consensus.cc:482] T 4d040ba420e246b6b1f6a736bde65803 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:27.595839 4444 raft_consensus.cc:3037] T 4d040ba420e246b6b1f6a736bde65803 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:27.597013 4444 raft_consensus.cc:504] T 4d040ba420e246b6b1f6a736bde65803 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:27.597342 4444 leader_election.cc:288] T 4d040ba420e246b6b1f6a736bde65803 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:27.597532 29295 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 20:46:27.599721 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d040ba420e246b6b1f6a736bde65803" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:27.599826 31888 raft_consensus.cc:3037] T 4d040ba420e246b6b1f6a736bde65803 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:27.600762 31888 raft_consensus.cc:2454] T 4d040ba420e246b6b1f6a736bde65803 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:27.601078 29249 leader_election.cc:302] T 4d040ba420e246b6b1f6a736bde65803 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:27.601243 4444 raft_consensus.cc:2781] T 4d040ba420e246b6b1f6a736bde65803 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:27.601310 4444 raft_consensus.cc:686] T 4d040ba420e246b6b1f6a736bde65803 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:27.601423 4444 consensus_queue.cc:237] T 4d040ba420e246b6b1f6a736bde65803 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:27.602783 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d040ba420e246b6b1f6a736bde65803" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:27.602885 31495 raft_consensus.cc:3037] T 4d040ba420e246b6b1f6a736bde65803 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:27.603533 29042 catalog_manager.cc:5484] T 4d040ba420e246b6b1f6a736bde65803 P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:27.603847 31495 raft_consensus.cc:2454] T 4d040ba420e246b6b1f6a736bde65803 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:27.604758 31475 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:27.608633 31605 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:27.614500 31735 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:27.622768 29163 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:27.631923 29556 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:27.641937 31867 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:27.657833 29425 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:27.660970 5278 ts_tablet_manager.cc:902] T b28fe6bccf4f48a6a5456e6ead5cc920 P 315915efcf2b413ca7f9d874338e1b2d: Initiating tablet copy from peer 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:27.661372 5278 tablet_copy_client.cc:308] T b28fe6bccf4f48a6a5456e6ead5cc920 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.65:43267
I20240502 20:46:27.661768 29202 tablet_copy_service.cc:140] P 7796ea3fba294d17a34386873969431d: Received BeginTabletCopySession request for tablet b28fe6bccf4f48a6a5456e6ead5cc920 from peer 315915efcf2b413ca7f9d874338e1b2d ({username='slave'} at 127.22.45.70:55755)
I20240502 20:46:27.661870 29202 tablet_copy_service.cc:161] P 7796ea3fba294d17a34386873969431d: Beginning new tablet copy session on tablet b28fe6bccf4f48a6a5456e6ead5cc920 from peer 315915efcf2b413ca7f9d874338e1b2d at {username='slave'} at 127.22.45.70:55755: session id = 315915efcf2b413ca7f9d874338e1b2d-b28fe6bccf4f48a6a5456e6ead5cc920
I20240502 20:46:27.662849 29202 tablet_copy_source_session.cc:205] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:27.664608 5278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b28fe6bccf4f48a6a5456e6ead5cc920. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:27.666849 5278 tablet_copy_client.cc:783] T b28fe6bccf4f48a6a5456e6ead5cc920 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:27.667129 5278 tablet_copy_client.cc:647] T b28fe6bccf4f48a6a5456e6ead5cc920 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:27.668223 5278 tablet_copy_client.cc:523] T b28fe6bccf4f48a6a5456e6ead5cc920 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:27.669239 5278 tablet_bootstrap.cc:492] T b28fe6bccf4f48a6a5456e6ead5cc920 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:27.679189 29042 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:27.686617 4944 tablet.cc:1722] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:27.687350 4678 tablet.cc:1722] T aa95224f07674eb28983ee58f319d168 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:27.691469 5214 tablet.cc:1722] T aa95224f07674eb28983ee58f319d168 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:27.691970 29035 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=787c6706bb3c4474950f26a0f0ee080d] alter complete (version 3)
I20240502 20:46:27.693439 5278 tablet_bootstrap.cc:492] T b28fe6bccf4f48a6a5456e6ead5cc920 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:27.693686 5278 tablet_bootstrap.cc:492] T b28fe6bccf4f48a6a5456e6ead5cc920 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap complete.
I20240502 20:46:27.693856 5278 ts_tablet_manager.cc:1367] T b28fe6bccf4f48a6a5456e6ead5cc920 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.025s user 0.019s sys 0.004s
I20240502 20:46:27.694532 5278 raft_consensus.cc:348] T b28fe6bccf4f48a6a5456e6ead5cc920 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:27.694648 5278 raft_consensus.cc:729] T b28fe6bccf4f48a6a5456e6ead5cc920 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:27.694768 5278 consensus_queue.cc:260] T b28fe6bccf4f48a6a5456e6ead5cc920 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:27.695108 5278 ts_tablet_manager.cc:1398] T b28fe6bccf4f48a6a5456e6ead5cc920 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 20:46:27.696252 29202 tablet_copy_service.cc:342] P 7796ea3fba294d17a34386873969431d: Request end of tablet copy session 315915efcf2b413ca7f9d874338e1b2d-b28fe6bccf4f48a6a5456e6ead5cc920 received from {username='slave'} at 127.22.45.70:55755
I20240502 20:46:27.696328 29202 tablet_copy_service.cc:433] P 7796ea3fba294d17a34386873969431d: ending tablet copy session 315915efcf2b413ca7f9d874338e1b2d-b28fe6bccf4f48a6a5456e6ead5cc920 on tablet b28fe6bccf4f48a6a5456e6ead5cc920 with peer 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:27.696352 4729 raft_consensus.cc:482] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:27.696449 4729 raft_consensus.cc:504] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:27.696835 4729 leader_election.cc:288] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:27.697149 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "370bf1bcfb3343c488954ece2488b4c2" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:27.697266 29175 raft_consensus.cc:2454] T 370bf1bcfb3343c488954ece2488b4c2 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:27.697566 29251 leader_election.cc:302] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:27.697626 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "370bf1bcfb3343c488954ece2488b4c2" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:27.697746 29577 raft_consensus.cc:2454] T 370bf1bcfb3343c488954ece2488b4c2 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:27.697813 4729 raft_consensus.cc:2781] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:27.697870 4729 raft_consensus.cc:482] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:27.697907 4729 raft_consensus.cc:3037] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:27.698840 4729 raft_consensus.cc:504] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:27.699193 4729 leader_election.cc:288] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:27.699394 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "370bf1bcfb3343c488954ece2488b4c2" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:27.699493 29577 raft_consensus.cc:3037] T 370bf1bcfb3343c488954ece2488b4c2 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:27.699510 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "370bf1bcfb3343c488954ece2488b4c2" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:27.699620 29175 raft_consensus.cc:3037] T 370bf1bcfb3343c488954ece2488b4c2 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:27.700340 29175 raft_consensus.cc:2454] T 370bf1bcfb3343c488954ece2488b4c2 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:27.700397 29577 raft_consensus.cc:2454] T 370bf1bcfb3343c488954ece2488b4c2 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:27.700625 29251 leader_election.cc:302] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:27.700804 4729 raft_consensus.cc:2781] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:27.700891 4729 raft_consensus.cc:686] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:27.701031 4729 consensus_queue.cc:237] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:27.703667 29038 catalog_manager.cc:5484] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:27.796100 29041 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:27.796227 29041 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:27.797916 29041 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet aa95224f07674eb28983ee58f319d168
I20240502 20:46:27.798453 31735 tablet_service.cc:1514] Processing DeleteTablet for tablet aa95224f07674eb28983ee58f319d168 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:27 UTC) from {username='slave'} at 127.0.0.1:33864
I20240502 20:46:27.798579 29163 tablet_service.cc:1514] Processing DeleteTablet for tablet aa95224f07674eb28983ee58f319d168 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:27 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:27.798605 31867 tablet_service.cc:1514] Processing DeleteTablet for tablet aa95224f07674eb28983ee58f319d168 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:27 UTC) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:27.798806 5326 tablet_replica.cc:330] T aa95224f07674eb28983ee58f319d168 P e7ab6b9053664ee5aa71c7978e26fe6a: stopping tablet replica
I20240502 20:46:27.798854 5327 tablet_replica.cc:330] T aa95224f07674eb28983ee58f319d168 P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:27.798964 5326 raft_consensus.cc:2229] T aa95224f07674eb28983ee58f319d168 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:27.799108 5327 raft_consensus.cc:2229] T aa95224f07674eb28983ee58f319d168 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:27.799178 5326 raft_consensus.cc:2258] T aa95224f07674eb28983ee58f319d168 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:27.799235 5328 tablet_replica.cc:330] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:27.799299 5327 raft_consensus.cc:2258] T aa95224f07674eb28983ee58f319d168 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:27.799404 5328 raft_consensus.cc:2229] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:27.799615 5328 raft_consensus.cc:2258] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:27.800027 5327 ts_tablet_manager.cc:1875] T aa95224f07674eb28983ee58f319d168 P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:27.800184 5326 ts_tablet_manager.cc:1875] T aa95224f07674eb28983ee58f319d168 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:27.800217 5328 ts_tablet_manager.cc:1875] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:27.802160 5326 ts_tablet_manager.cc:1888] T aa95224f07674eb28983ee58f319d168 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:27.802228 5327 ts_tablet_manager.cc:1888] T aa95224f07674eb28983ee58f319d168 P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:27.802228 5328 ts_tablet_manager.cc:1888] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:27.802263 5326 log.cc:1196] T aa95224f07674eb28983ee58f319d168 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/wals/aa95224f07674eb28983ee58f319d168
I20240502 20:46:27.802302 5328 log.cc:1196] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/aa95224f07674eb28983ee58f319d168
I20240502 20:46:27.802330 5327 log.cc:1196] T aa95224f07674eb28983ee58f319d168 P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/aa95224f07674eb28983ee58f319d168
I20240502 20:46:27.802642 5326 ts_tablet_manager.cc:1909] T aa95224f07674eb28983ee58f319d168 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting consensus metadata
I20240502 20:46:27.802664 5327 ts_tablet_manager.cc:1909] T aa95224f07674eb28983ee58f319d168 P e6b60803d0764d228c863e2da65e3691: Deleting consensus metadata
I20240502 20:46:27.802664 5328 ts_tablet_manager.cc:1909] T aa95224f07674eb28983ee58f319d168 P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:27.803272 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet aa95224f07674eb28983ee58f319d168 (table rebalancer_dynamic_table_new_name [id=787c6706bb3c4474950f26a0f0ee080d]) successfully deleted
I20240502 20:46:27.803360 29026 catalog_manager.cc:4830] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): tablet aa95224f07674eb28983ee58f319d168 (table rebalancer_dynamic_table_new_name [id=787c6706bb3c4474950f26a0f0ee080d]) successfully deleted
I20240502 20:46:27.803444 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet aa95224f07674eb28983ee58f319d168 (table rebalancer_dynamic_table_new_name [id=787c6706bb3c4474950f26a0f0ee080d]) successfully deleted
I20240502 20:46:27.895103 4729 raft_consensus.cc:482] T 9992a7ea196645dc82d2430fcb405254 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:27.895215 4729 raft_consensus.cc:504] T 9992a7ea196645dc82d2430fcb405254 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:27.895648 4729 leader_election.cc:288] T 9992a7ea196645dc82d2430fcb405254 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:27.895962 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9992a7ea196645dc82d2430fcb405254" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:27.896042 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9992a7ea196645dc82d2430fcb405254" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:27.896113 29574 raft_consensus.cc:2454] T 9992a7ea196645dc82d2430fcb405254 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:27.896183 31625 raft_consensus.cc:2454] T 9992a7ea196645dc82d2430fcb405254 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:27.896425 29249 leader_election.cc:302] T 9992a7ea196645dc82d2430fcb405254 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:27.896690 4729 raft_consensus.cc:2781] T 9992a7ea196645dc82d2430fcb405254 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:27.896739 4729 raft_consensus.cc:482] T 9992a7ea196645dc82d2430fcb405254 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:27.896777 4729 raft_consensus.cc:3037] T 9992a7ea196645dc82d2430fcb405254 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:27.897801 4729 raft_consensus.cc:504] T 9992a7ea196645dc82d2430fcb405254 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:27.898131 4729 leader_election.cc:288] T 9992a7ea196645dc82d2430fcb405254 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:27.898576 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9992a7ea196645dc82d2430fcb405254" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:27.898579 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9992a7ea196645dc82d2430fcb405254" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:27.898676 31625 raft_consensus.cc:3037] T 9992a7ea196645dc82d2430fcb405254 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:27.898680 29574 raft_consensus.cc:3037] T 9992a7ea196645dc82d2430fcb405254 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:27.899659 31625 raft_consensus.cc:2454] T 9992a7ea196645dc82d2430fcb405254 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:27.899724 29574 raft_consensus.cc:2454] T 9992a7ea196645dc82d2430fcb405254 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:27.899940 29041 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:27.900017 29249 leader_election.cc:302] T 9992a7ea196645dc82d2430fcb405254 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 315915efcf2b413ca7f9d874338e1b2d; no voters:
I20240502 20:46:27.900250 4729 raft_consensus.cc:2781] T 9992a7ea196645dc82d2430fcb405254 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:27.900332 4729 raft_consensus.cc:686] T 9992a7ea196645dc82d2430fcb405254 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:27.900456 4729 consensus_queue.cc:237] T 9992a7ea196645dc82d2430fcb405254 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:27.905988 29033 catalog_manager.cc:5484] T 9992a7ea196645dc82d2430fcb405254 P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:27.906657 31735 tablet_service.cc:1467] Processing CreateTablet for tablet 3a12c5ddc1804c0d90420c1e394af429 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=aec5ec7a3fe248f3be5309e35917b762]), partition=
I20240502 20:46:27.906792 29425 tablet_service.cc:1467] Processing CreateTablet for tablet 3a12c5ddc1804c0d90420c1e394af429 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=aec5ec7a3fe248f3be5309e35917b762]), partition=
I20240502 20:46:27.906908 31735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a12c5ddc1804c0d90420c1e394af429. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:27.906986 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a12c5ddc1804c0d90420c1e394af429. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:27.909111 5331 tablet_bootstrap.cc:492] T 3a12c5ddc1804c0d90420c1e394af429 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:27.909459 31605 tablet_service.cc:1467] Processing CreateTablet for tablet 3a12c5ddc1804c0d90420c1e394af429 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=aec5ec7a3fe248f3be5309e35917b762]), partition=
I20240502 20:46:27.909686 31605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a12c5ddc1804c0d90420c1e394af429. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:27.909771 5330 tablet_bootstrap.cc:492] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:27.909947 5331 tablet_bootstrap.cc:654] T 3a12c5ddc1804c0d90420c1e394af429 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:27.910845 5330 tablet_bootstrap.cc:654] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:27.911127 5331 tablet_bootstrap.cc:492] T 3a12c5ddc1804c0d90420c1e394af429 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:27.911201 5331 ts_tablet_manager.cc:1367] T 3a12c5ddc1804c0d90420c1e394af429 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:27.911681 5254 tablet_bootstrap.cc:492] T 3a12c5ddc1804c0d90420c1e394af429 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:27.911629 5331 raft_consensus.cc:348] T 3a12c5ddc1804c0d90420c1e394af429 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:27.911725 5331 raft_consensus.cc:374] T 3a12c5ddc1804c0d90420c1e394af429 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:27.911757 5331 raft_consensus.cc:729] T 3a12c5ddc1804c0d90420c1e394af429 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:27.911861 5331 consensus_queue.cc:260] T 3a12c5ddc1804c0d90420c1e394af429 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:27.912034 5330 tablet_bootstrap.cc:492] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:27.912139 5331 ts_tablet_manager.cc:1398] T 3a12c5ddc1804c0d90420c1e394af429 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:27.912155 5330 ts_tablet_manager.cc:1367] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:27.912508 5254 tablet_bootstrap.cc:654] T 3a12c5ddc1804c0d90420c1e394af429 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:27.912824 5330 raft_consensus.cc:348] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:27.912917 5330 raft_consensus.cc:374] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:27.912945 5330 raft_consensus.cc:729] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:27.913182 5330 consensus_queue.cc:260] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:27.913550 5330 ts_tablet_manager.cc:1398] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:27.913940 5254 tablet_bootstrap.cc:492] T 3a12c5ddc1804c0d90420c1e394af429 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:27.914021 5254 ts_tablet_manager.cc:1367] T 3a12c5ddc1804c0d90420c1e394af429 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:27.914419 5254 raft_consensus.cc:348] T 3a12c5ddc1804c0d90420c1e394af429 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:27.914543 5254 raft_consensus.cc:374] T 3a12c5ddc1804c0d90420c1e394af429 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:27.914585 5254 raft_consensus.cc:729] T 3a12c5ddc1804c0d90420c1e394af429 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:27.914686 5254 consensus_queue.cc:260] T 3a12c5ddc1804c0d90420c1e394af429 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:27.914978 5254 ts_tablet_manager.cc:1398] T 3a12c5ddc1804c0d90420c1e394af429 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:27.942669 31605 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:27.943761 29556 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:27.951045 31475 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:27.952013 29163 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:27.952257 29425 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:27.955451 31735 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:27.959224 31867 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:27.967830 31495 raft_consensus.cc:1203] T 849c31557dec4043ad36dbddc6811b3f P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:27.999888 29295 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:28.037856 4729 consensus_queue.cc:1035] T 4d040ba420e246b6b1f6a736bde65803 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:28.041749 29446 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "21ee3d7c6f764d0483b335da40e61efc"
dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:46582
I20240502 20:46:28.041878 29446 raft_consensus.cc:595] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Received request to transfer leadership
I20240502 20:46:28.044520 4444 consensus_queue.cc:1035] T 4d040ba420e246b6b1f6a736bde65803 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:28.072943 31755 raft_consensus.cc:1203] T 035762cba9014632ad520f4777544641 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:28.123396 5199 raft_consensus.cc:979] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6: : Instructing follower 2a9c1af8cf814030bde56b67ca5c2fbe to start an election
I20240502 20:46:28.123513 5199 raft_consensus.cc:1067] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Signalling peer 2a9c1af8cf814030bde56b67ca5c2fbe to start an election
I20240502 20:46:28.123925 31495 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "21ee3d7c6f764d0483b335da40e61efc"
dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
from {username='slave'} at 127.22.45.67:46915
I20240502 20:46:28.124065 31495 raft_consensus.cc:482] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:28.124111 31495 raft_consensus.cc:3037] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:28.124954 31495 raft_consensus.cc:504] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:28.125512 31495 leader_election.cc:288] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 2 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:28.125783 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ee3d7c6f764d0483b335da40e61efc" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:28.125866 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ee3d7c6f764d0483b335da40e61efc" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:28.125929 29175 raft_consensus.cc:3037] T 21ee3d7c6f764d0483b335da40e61efc P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:28.125977 29315 raft_consensus.cc:3037] T 21ee3d7c6f764d0483b335da40e61efc P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:28.126991 29175 raft_consensus.cc:2454] T 21ee3d7c6f764d0483b335da40e61efc P 7796ea3fba294d17a34386873969431d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 2.
I20240502 20:46:28.126986 29315 raft_consensus.cc:2454] T 21ee3d7c6f764d0483b335da40e61efc P 09258ac095664fa99caf0707262cde13 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 2.
I20240502 20:46:28.128422 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ee3d7c6f764d0483b335da40e61efc" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:28.128451 31431 leader_election.cc:302] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 2a9c1af8cf814030bde56b67ca5c2fbe, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:28.128537 29446 raft_consensus.cc:3032] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:28.128592 29446 raft_consensus.cc:729] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:28.128628 4975 raft_consensus.cc:2781] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:28.128713 4975 raft_consensus.cc:686] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe [term 2 LEADER]: Becoming Leader. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Running, Role: LEADER
I20240502 20:46:28.128733 29446 consensus_queue.cc:260] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:28.128845 4975 consensus_queue.cc:237] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:28.128948 29446 raft_consensus.cc:3037] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:28.129981 29446 raft_consensus.cc:2454] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 2.
I20240502 20:46:28.131001 29042 catalog_manager.cc:5484] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: term changed from 1 to 2, leader changed from dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) to 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69). New cstate: current_term: 2 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:28.142118 5199 raft_consensus.cc:482] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:28.142210 5199 raft_consensus.cc:504] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:28.142637 5199 leader_election.cc:288] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:28.142930 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a12c5ddc1804c0d90420c1e394af429" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:28.143049 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a12c5ddc1804c0d90420c1e394af429" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:28.143088 31755 raft_consensus.cc:2454] T 3a12c5ddc1804c0d90420c1e394af429 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:28.143174 31625 raft_consensus.cc:2454] T 3a12c5ddc1804c0d90420c1e394af429 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:28.143457 29380 leader_election.cc:302] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:28.143625 5199 raft_consensus.cc:2781] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:28.143683 5199 raft_consensus.cc:482] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:28.143746 5199 raft_consensus.cc:3037] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:28.144711 5199 raft_consensus.cc:504] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:28.145043 5199 leader_election.cc:288] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:28.145385 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a12c5ddc1804c0d90420c1e394af429" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:28.145501 31757 raft_consensus.cc:3037] T 3a12c5ddc1804c0d90420c1e394af429 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:28.146369 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a12c5ddc1804c0d90420c1e394af429" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:28.146517 31757 raft_consensus.cc:2454] T 3a12c5ddc1804c0d90420c1e394af429 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:28.146572 31625 raft_consensus.cc:3037] T 3a12c5ddc1804c0d90420c1e394af429 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:28.147464 31625 raft_consensus.cc:2454] T 3a12c5ddc1804c0d90420c1e394af429 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:28.147697 29380 leader_election.cc:302] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:28.147835 4694 raft_consensus.cc:2781] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:28.147925 4694 raft_consensus.cc:686] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:28.148056 4694 consensus_queue.cc:237] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:28.149809 29042 catalog_manager.cc:5484] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:28.169871 31605 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:28.172005 31475 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:28.173751 29163 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:28.176720 31625 raft_consensus.cc:1203] T b28fe6bccf4f48a6a5456e6ead5cc920 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:28.182780 29556 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:28.189299 29425 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:28.193228 31867 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:28.195943 31735 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:28.200881 4444 consensus_queue.cc:1035] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:28.203055 29295 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:28.212821 4444 consensus_queue.cc:1035] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:28.254213 29042 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:28.258452 31625 raft_consensus.cc:1261] T 3a12c5ddc1804c0d90420c1e394af429 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:28.258556 31757 raft_consensus.cc:1261] T 3a12c5ddc1804c0d90420c1e394af429 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:28.258873 5199 consensus_queue.cc:1035] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:28.259011 4694 consensus_queue.cc:1035] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:28.262440 29042 catalog_manager.cc:5576] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751) does not have the latest schema for tablet 3a12c5ddc1804c0d90420c1e394af429 (table rebalancer_dynamic_table [id=aec5ec7a3fe248f3be5309e35917b762]). Expected version 1 got 0
I20240502 20:46:28.263357 29039 catalog_manager.cc:5576] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787) does not have the latest schema for tablet 3a12c5ddc1804c0d90420c1e394af429 (table rebalancer_dynamic_table [id=aec5ec7a3fe248f3be5309e35917b762]). Expected version 1 got 0
I20240502 20:46:28.263336 5427 tablet.cc:1722] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:28.263638 5427 tablet.cc:1620] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6: MemRowSet was empty: no flush needed.
I20240502 20:46:28.264762 5428 tablet.cc:1722] T 3a12c5ddc1804c0d90420c1e394af429 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:28.265285 5428 tablet.cc:1620] T 3a12c5ddc1804c0d90420c1e394af429 P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:28.266093 29039 catalog_manager.cc:5576] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787) does not have the latest schema for tablet 3a12c5ddc1804c0d90420c1e394af429 (table rebalancer_dynamic_table [id=aec5ec7a3fe248f3be5309e35917b762]). Expected version 1 got 0
I20240502 20:46:28.269157 29039 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=aec5ec7a3fe248f3be5309e35917b762] alter complete (version 1)
I20240502 20:46:28.272969 5429 tablet.cc:1722] T 3a12c5ddc1804c0d90420c1e394af429 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:28.273928 5429 tablet.cc:1620] T 3a12c5ddc1804c0d90420c1e394af429 P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:28.274264 29039 catalog_manager.cc:5576] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751) does not have the latest schema for tablet 3a12c5ddc1804c0d90420c1e394af429 (table rebalancer_dynamic_table [id=aec5ec7a3fe248f3be5309e35917b762]). Expected version 1 got 0
I20240502 20:46:28.345311 31475 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:28.355376 29425 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:28.356695 29163 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:28.360250 31605 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:28.363309 29295 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:28.371831 31867 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:28.374118 29036 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:28.383584 5427 tablet.cc:1722] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:28.383883 5427 tablet.cc:1620] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6: MemRowSet was empty: no flush needed.
I20240502 20:46:28.385131 5429 tablet.cc:1722] T 3a12c5ddc1804c0d90420c1e394af429 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:28.385396 5429 tablet.cc:1620] T 3a12c5ddc1804c0d90420c1e394af429 P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:28.389858 31735 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:28.393057 5428 tablet.cc:1722] T 3a12c5ddc1804c0d90420c1e394af429 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:28.393368 5428 tablet.cc:1620] T 3a12c5ddc1804c0d90420c1e394af429 P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:28.393864 29034 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=aec5ec7a3fe248f3be5309e35917b762] alter complete (version 2)
I20240502 20:46:28.397758 29556 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:28.434780 29040 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_42"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:28.441749 31475 tablet_service.cc:1467] Processing CreateTablet for tablet d75abb6dc8a247d796194acad2d25edd (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=736e150fd2e24fcfad45c5cba0b31c35]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:28.441939 29163 tablet_service.cc:1467] Processing CreateTablet for tablet d75abb6dc8a247d796194acad2d25edd (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=736e150fd2e24fcfad45c5cba0b31c35]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:28.442027 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d75abb6dc8a247d796194acad2d25edd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:28.442179 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d75abb6dc8a247d796194acad2d25edd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:28.442521 29162 tablet_service.cc:1467] Processing CreateTablet for tablet 8705260a889841bfa515683432dfe99d (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=736e150fd2e24fcfad45c5cba0b31c35]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:28.442773 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8705260a889841bfa515683432dfe99d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:28.443051 29425 tablet_service.cc:1467] Processing CreateTablet for tablet 28d60f6c42e446a1bc2bc9ea4c7e55b8 (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=736e150fd2e24fcfad45c5cba0b31c35]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:28.443287 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28d60f6c42e446a1bc2bc9ea4c7e55b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:28.444540 29556 tablet_service.cc:1467] Processing CreateTablet for tablet 28d60f6c42e446a1bc2bc9ea4c7e55b8 (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=736e150fd2e24fcfad45c5cba0b31c35]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:28.444787 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28d60f6c42e446a1bc2bc9ea4c7e55b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:28.445061 5476 tablet_bootstrap.cc:492] T d75abb6dc8a247d796194acad2d25edd P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:28.445281 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 28d60f6c42e446a1bc2bc9ea4c7e55b8 (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=736e150fd2e24fcfad45c5cba0b31c35]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:28.445534 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28d60f6c42e446a1bc2bc9ea4c7e55b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:28.446040 5476 tablet_bootstrap.cc:654] T d75abb6dc8a247d796194acad2d25edd P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:28.446782 5477 tablet_bootstrap.cc:492] T d75abb6dc8a247d796194acad2d25edd P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:28.447163 31735 tablet_service.cc:1467] Processing CreateTablet for tablet d75abb6dc8a247d796194acad2d25edd (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=736e150fd2e24fcfad45c5cba0b31c35]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:28.447418 31735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d75abb6dc8a247d796194acad2d25edd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:28.447491 5478 tablet_bootstrap.cc:492] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:28.447840 31868 tablet_service.cc:1467] Processing CreateTablet for tablet 8705260a889841bfa515683432dfe99d (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=736e150fd2e24fcfad45c5cba0b31c35]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:28.448060 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8705260a889841bfa515683432dfe99d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:28.448282 5477 tablet_bootstrap.cc:654] T d75abb6dc8a247d796194acad2d25edd P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:28.448396 5478 tablet_bootstrap.cc:654] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:28.448688 5479 tablet_bootstrap.cc:492] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:28.449183 5480 tablet_bootstrap.cc:492] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:28.450250 31735 tablet_service.cc:1467] Processing CreateTablet for tablet 8705260a889841bfa515683432dfe99d (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=736e150fd2e24fcfad45c5cba0b31c35]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:28.450358 5480 tablet_bootstrap.cc:654] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:28.450480 31735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8705260a889841bfa515683432dfe99d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:28.450564 5479 tablet_bootstrap.cc:654] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:28.451375 5476 tablet_bootstrap.cc:492] T d75abb6dc8a247d796194acad2d25edd P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:28.451481 5476 ts_tablet_manager.cc:1367] T d75abb6dc8a247d796194acad2d25edd P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 20:46:28.451875 5478 tablet_bootstrap.cc:492] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:28.452034 5478 ts_tablet_manager.cc:1367] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 20:46:28.452311 5476 raft_consensus.cc:348] T d75abb6dc8a247d796194acad2d25edd P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:28.452502 5478 raft_consensus.cc:348] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:28.452555 5476 raft_consensus.cc:374] T d75abb6dc8a247d796194acad2d25edd P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:28.452606 5476 raft_consensus.cc:729] T d75abb6dc8a247d796194acad2d25edd P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:28.452608 5478 raft_consensus.cc:374] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:28.452639 5478 raft_consensus.cc:729] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:28.452800 5476 consensus_queue.cc:260] T d75abb6dc8a247d796194acad2d25edd P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:28.453030 5479 tablet_bootstrap.cc:492] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:28.453230 5483 tablet_bootstrap.cc:492] T d75abb6dc8a247d796194acad2d25edd P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:28.453217 5478 consensus_queue.cc:260] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:28.453590 5478 ts_tablet_manager.cc:1398] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:28.453727 5479 ts_tablet_manager.cc:1367] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20240502 20:46:28.453832 5476 ts_tablet_manager.cc:1398] T d75abb6dc8a247d796194acad2d25edd P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:28.454058 5476 tablet_bootstrap.cc:492] T 8705260a889841bfa515683432dfe99d P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:28.454176 5479 raft_consensus.cc:348] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:28.454279 5479 raft_consensus.cc:374] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:28.454309 5479 raft_consensus.cc:729] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:28.454334 5483 tablet_bootstrap.cc:654] T d75abb6dc8a247d796194acad2d25edd P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:28.454594 5479 consensus_queue.cc:260] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:28.454957 5479 ts_tablet_manager.cc:1398] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:28.455787 5483 tablet_bootstrap.cc:492] T d75abb6dc8a247d796194acad2d25edd P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:28.455886 5483 ts_tablet_manager.cc:1367] T d75abb6dc8a247d796194acad2d25edd P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:28.456339 5483 raft_consensus.cc:348] T d75abb6dc8a247d796194acad2d25edd P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:28.456662 5483 raft_consensus.cc:374] T d75abb6dc8a247d796194acad2d25edd P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:28.456743 5483 raft_consensus.cc:729] T d75abb6dc8a247d796194acad2d25edd P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:28.456873 5483 consensus_queue.cc:260] T d75abb6dc8a247d796194acad2d25edd P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:28.457211 5483 ts_tablet_manager.cc:1398] T d75abb6dc8a247d796194acad2d25edd P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:28.459093 5483 tablet_bootstrap.cc:492] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:28.461568 5483 tablet_bootstrap.cc:654] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:28.463068 5477 tablet_bootstrap.cc:492] T d75abb6dc8a247d796194acad2d25edd P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:28.463213 5477 ts_tablet_manager.cc:1367] T d75abb6dc8a247d796194acad2d25edd P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240502 20:46:28.463512 5483 tablet_bootstrap.cc:492] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:28.463598 5483 ts_tablet_manager.cc:1367] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 20:46:28.463883 5477 raft_consensus.cc:348] T d75abb6dc8a247d796194acad2d25edd P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:28.463997 5477 raft_consensus.cc:374] T d75abb6dc8a247d796194acad2d25edd P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:28.464032 5477 raft_consensus.cc:729] T d75abb6dc8a247d796194acad2d25edd P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:28.464010 5483 raft_consensus.cc:348] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:28.464217 5477 consensus_queue.cc:260] T d75abb6dc8a247d796194acad2d25edd P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:28.464419 5476 tablet_bootstrap.cc:654] T 8705260a889841bfa515683432dfe99d P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:28.464583 5477 ts_tablet_manager.cc:1398] T d75abb6dc8a247d796194acad2d25edd P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:28.464848 5483 raft_consensus.cc:374] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:28.464896 5483 raft_consensus.cc:729] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:28.465013 5483 consensus_queue.cc:260] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:28.465267 5483 ts_tablet_manager.cc:1398] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:28.465384 5476 tablet_bootstrap.cc:492] T 8705260a889841bfa515683432dfe99d P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:28.465456 5476 ts_tablet_manager.cc:1367] T 8705260a889841bfa515683432dfe99d P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240502 20:46:28.465767 5480 tablet_bootstrap.cc:492] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:28.465854 5480 ts_tablet_manager.cc:1367] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.001s
I20240502 20:46:28.465883 5476 raft_consensus.cc:348] T 8705260a889841bfa515683432dfe99d P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:28.465983 5476 raft_consensus.cc:374] T 8705260a889841bfa515683432dfe99d P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:28.466015 5476 raft_consensus.cc:729] T 8705260a889841bfa515683432dfe99d P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:28.466154 5476 consensus_queue.cc:260] T 8705260a889841bfa515683432dfe99d P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:28.466431 5476 ts_tablet_manager.cc:1398] T 8705260a889841bfa515683432dfe99d P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:28.466270 5480 raft_consensus.cc:348] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:28.466965 5480 raft_consensus.cc:374] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:28.467012 5480 raft_consensus.cc:729] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:28.467125 5480 consensus_queue.cc:260] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:28.470134 5480 ts_tablet_manager.cc:1398] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20240502 20:46:28.470340 5480 tablet_bootstrap.cc:492] T 8705260a889841bfa515683432dfe99d P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:28.471258 5480 tablet_bootstrap.cc:654] T 8705260a889841bfa515683432dfe99d P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:28.472908 5480 tablet_bootstrap.cc:492] T 8705260a889841bfa515683432dfe99d P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:28.473008 5480 ts_tablet_manager.cc:1367] T 8705260a889841bfa515683432dfe99d P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:28.473543 5480 raft_consensus.cc:348] T 8705260a889841bfa515683432dfe99d P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:28.473647 5480 raft_consensus.cc:374] T 8705260a889841bfa515683432dfe99d P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:28.473676 5480 raft_consensus.cc:729] T 8705260a889841bfa515683432dfe99d P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:28.473786 5480 consensus_queue.cc:260] T 8705260a889841bfa515683432dfe99d P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:28.474053 5480 ts_tablet_manager.cc:1398] T 8705260a889841bfa515683432dfe99d P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:28.481449 29574 consensus_queue.cc:237] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:28.482581 31755 raft_consensus.cc:1261] T 9acc12c9f0464e07965c05285fd334f0 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:28.482724 29446 raft_consensus.cc:1261] T 9acc12c9f0464e07965c05285fd334f0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:28.482932 4216 consensus_queue.cc:1035] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:28.483137 5346 consensus_queue.cc:1035] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:28.484650 4216 raft_consensus.cc:2932] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:28.485073 31755 raft_consensus.cc:2932] T 9acc12c9f0464e07965c05285fd334f0 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:28.486999 29446 raft_consensus.cc:2932] T 9acc12c9f0464e07965c05285fd334f0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:28.487644 29033 catalog_manager.cc:5484] T 9acc12c9f0464e07965c05285fd334f0 P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: config changed from index 4 to 5, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) evicted. New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:28.490919 29295 tablet_service.cc:1514] Processing DeleteTablet for tablet 9acc12c9f0464e07965c05285fd334f0 with delete_type TABLET_DATA_TOMBSTONED (TS 09258ac095664fa99caf0707262cde13 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:28.491282 5506 tablet_replica.cc:330] T 9acc12c9f0464e07965c05285fd334f0 P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:28.491479 5506 raft_consensus.cc:2229] T 9acc12c9f0464e07965c05285fd334f0 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:28.491706 5506 raft_consensus.cc:2258] T 9acc12c9f0464e07965c05285fd334f0 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:28.492277 5506 ts_tablet_manager.cc:1875] T 9acc12c9f0464e07965c05285fd334f0 P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:28.492874 4729 consensus_queue.cc:1035] T 9992a7ea196645dc82d2430fcb405254 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:28.495231 5506 ts_tablet_manager.cc:1888] T 9acc12c9f0464e07965c05285fd334f0 P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 20:46:28.495309 5506 log.cc:1196] T 9acc12c9f0464e07965c05285fd334f0 P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/9acc12c9f0464e07965c05285fd334f0
I20240502 20:46:28.495822 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet 9acc12c9f0464e07965c05285fd334f0 (table rebalance_test_table_9 [id=4b510e9cf32a4794a9193f7ebac46f93]) successfully deleted
I20240502 20:46:28.496125 29036 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:28.496296 4729 consensus_queue.cc:1035] T 9992a7ea196645dc82d2430fcb405254 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:28.503414 5427 tablet.cc:1722] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:28.504137 5428 tablet.cc:1722] T 3a12c5ddc1804c0d90420c1e394af429 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:28.506707 5429 tablet.cc:1722] T 3a12c5ddc1804c0d90420c1e394af429 P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:28.508831 29036 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=aec5ec7a3fe248f3be5309e35917b762] alter complete (version 3)
I20240502 20:46:28.537499 3289 raft_consensus.cc:482] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:28.537604 3289 raft_consensus.cc:504] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:28.537979 3289 leader_election.cc:288] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:28.538362 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8705260a889841bfa515683432dfe99d" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:28.538524 29175 raft_consensus.cc:2454] T 8705260a889841bfa515683432dfe99d P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:28.538540 31887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8705260a889841bfa515683432dfe99d" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:28.538666 31887 raft_consensus.cc:2454] T 8705260a889841bfa515683432dfe99d P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:28.538977 31693 leader_election.cc:302] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:28.539278 3289 raft_consensus.cc:2781] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:28.539333 3289 raft_consensus.cc:482] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:28.539371 3289 raft_consensus.cc:3037] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:28.540354 3289 raft_consensus.cc:504] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:28.540676 3289 leader_election.cc:288] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:28.541563 31887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8705260a889841bfa515683432dfe99d" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:28.541680 31887 raft_consensus.cc:3037] T 8705260a889841bfa515683432dfe99d P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:28.541661 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8705260a889841bfa515683432dfe99d" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:28.541756 29175 raft_consensus.cc:3037] T 8705260a889841bfa515683432dfe99d P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:28.542639 31887 raft_consensus.cc:2454] T 8705260a889841bfa515683432dfe99d P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:28.543040 29175 raft_consensus.cc:2454] T 8705260a889841bfa515683432dfe99d P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:28.543340 31691 leader_election.cc:302] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e6b60803d0764d228c863e2da65e3691, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:28.543777 5252 raft_consensus.cc:2781] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:28.543987 5252 raft_consensus.cc:686] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Becoming Leader. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Running, Role: LEADER
I20240502 20:46:28.544132 5252 consensus_queue.cc:237] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:28.546772 29040 catalog_manager.cc:5484] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: term changed from 0 to 1, leader changed from <none> to e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71). New cstate: current_term: 1 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:28.611883 29036 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:28.612033 29036 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:28.614233 29036 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 3a12c5ddc1804c0d90420c1e394af429
I20240502 20:46:28.614763 31735 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a12c5ddc1804c0d90420c1e394af429 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:28 UTC) from {username='slave'} at 127.0.0.1:33864
I20240502 20:46:28.614779 31605 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a12c5ddc1804c0d90420c1e394af429 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:28 UTC) from {username='slave'} at 127.0.0.1:58346
I20240502 20:46:28.615077 5553 tablet_replica.cc:330] T 3a12c5ddc1804c0d90420c1e394af429 P e7ab6b9053664ee5aa71c7978e26fe6a: stopping tablet replica
I20240502 20:46:28.615427 29425 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a12c5ddc1804c0d90420c1e394af429 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:28 UTC) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:28.615685 5553 raft_consensus.cc:2229] T 3a12c5ddc1804c0d90420c1e394af429 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:28.615723 5554 tablet_replica.cc:330] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:28.615850 5554 raft_consensus.cc:2229] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:28.615901 5553 raft_consensus.cc:2258] T 3a12c5ddc1804c0d90420c1e394af429 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:28.616094 5554 raft_consensus.cc:2258] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:28.616585 5553 ts_tablet_manager.cc:1875] T 3a12c5ddc1804c0d90420c1e394af429 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:28.616719 5554 ts_tablet_manager.cc:1875] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:28.618880 5554 ts_tablet_manager.cc:1888] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:28.618881 5553 ts_tablet_manager.cc:1888] T 3a12c5ddc1804c0d90420c1e394af429 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:28.618948 5553 log.cc:1196] T 3a12c5ddc1804c0d90420c1e394af429 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/wals/3a12c5ddc1804c0d90420c1e394af429
I20240502 20:46:28.619014 5554 log.cc:1196] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/3a12c5ddc1804c0d90420c1e394af429
I20240502 20:46:28.619254 5553 ts_tablet_manager.cc:1909] T 3a12c5ddc1804c0d90420c1e394af429 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting consensus metadata
I20240502 20:46:28.619401 5554 ts_tablet_manager.cc:1909] T 3a12c5ddc1804c0d90420c1e394af429 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting consensus metadata
I20240502 20:46:28.619819 29026 catalog_manager.cc:4830] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): tablet 3a12c5ddc1804c0d90420c1e394af429 (table rebalancer_dynamic_table_new_name [id=aec5ec7a3fe248f3be5309e35917b762]) successfully deleted
I20240502 20:46:28.620093 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet 3a12c5ddc1804c0d90420c1e394af429 (table rebalancer_dynamic_table_new_name [id=aec5ec7a3fe248f3be5309e35917b762]) successfully deleted
I20240502 20:46:28.621690 29446 raft_consensus.cc:1261] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 2 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:28.622032 4975 consensus_queue.cc:1035] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:28.626719 5552 tablet_replica.cc:330] T 3a12c5ddc1804c0d90420c1e394af429 P 315915efcf2b413ca7f9d874338e1b2d: stopping tablet replica
I20240502 20:46:28.626865 5552 raft_consensus.cc:2229] T 3a12c5ddc1804c0d90420c1e394af429 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:28.626976 5552 raft_consensus.cc:2258] T 3a12c5ddc1804c0d90420c1e394af429 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:28.627462 5552 ts_tablet_manager.cc:1875] T 3a12c5ddc1804c0d90420c1e394af429 P 315915efcf2b413ca7f9d874338e1b2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:28.629179 5552 ts_tablet_manager.cc:1888] T 3a12c5ddc1804c0d90420c1e394af429 P 315915efcf2b413ca7f9d874338e1b2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:28.629237 5552 log.cc:1196] T 3a12c5ddc1804c0d90420c1e394af429 P 315915efcf2b413ca7f9d874338e1b2d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal/wals/3a12c5ddc1804c0d90420c1e394af429
I20240502 20:46:28.629410 5552 ts_tablet_manager.cc:1909] T 3a12c5ddc1804c0d90420c1e394af429 P 315915efcf2b413ca7f9d874338e1b2d: Deleting consensus metadata
I20240502 20:46:28.629976 29027 catalog_manager.cc:4830] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): tablet 3a12c5ddc1804c0d90420c1e394af429 (table rebalancer_dynamic_table_new_name [id=aec5ec7a3fe248f3be5309e35917b762]) successfully deleted
I20240502 20:46:28.658867 29175 raft_consensus.cc:1261] T 21ee3d7c6f764d0483b335da40e61efc P 7796ea3fba294d17a34386873969431d [term 2 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:28.659238 4975 consensus_queue.cc:1035] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:28.665608 5032 raft_consensus.cc:482] T d75abb6dc8a247d796194acad2d25edd P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:28.665709 5032 raft_consensus.cc:504] T d75abb6dc8a247d796194acad2d25edd P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:28.666101 5032 leader_election.cc:288] T d75abb6dc8a247d796194acad2d25edd P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:28.666358 29315 raft_consensus.cc:1261] T 21ee3d7c6f764d0483b335da40e61efc P 09258ac095664fa99caf0707262cde13 [term 2 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:28.666745 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d75abb6dc8a247d796194acad2d25edd" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:28.666889 31755 raft_consensus.cc:2454] T d75abb6dc8a247d796194acad2d25edd P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:28.667140 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d75abb6dc8a247d796194acad2d25edd" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:28.667204 29117 leader_election.cc:302] T d75abb6dc8a247d796194acad2d25edd P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:28.667261 31495 raft_consensus.cc:2454] T d75abb6dc8a247d796194acad2d25edd P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:28.667481 5558 consensus_queue.cc:1035] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:28.667589 5033 raft_consensus.cc:2781] T d75abb6dc8a247d796194acad2d25edd P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:28.667647 5033 raft_consensus.cc:482] T d75abb6dc8a247d796194acad2d25edd P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:28.667694 5033 raft_consensus.cc:3037] T d75abb6dc8a247d796194acad2d25edd P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:28.668536 5033 raft_consensus.cc:504] T d75abb6dc8a247d796194acad2d25edd P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:28.668874 5033 leader_election.cc:288] T d75abb6dc8a247d796194acad2d25edd P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:28.669188 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d75abb6dc8a247d796194acad2d25edd" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:28.669297 31755 raft_consensus.cc:3037] T d75abb6dc8a247d796194acad2d25edd P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:28.669620 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d75abb6dc8a247d796194acad2d25edd" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:28.669723 31495 raft_consensus.cc:3037] T d75abb6dc8a247d796194acad2d25edd P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:28.670269 31755 raft_consensus.cc:2454] T d75abb6dc8a247d796194acad2d25edd P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:28.670722 31495 raft_consensus.cc:2454] T d75abb6dc8a247d796194acad2d25edd P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:28.671077 29117 leader_election.cc:302] T d75abb6dc8a247d796194acad2d25edd P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:28.671232 5033 raft_consensus.cc:2781] T d75abb6dc8a247d796194acad2d25edd P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:28.671303 5033 raft_consensus.cc:686] T d75abb6dc8a247d796194acad2d25edd P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:28.671422 5033 consensus_queue.cc:237] T d75abb6dc8a247d796194acad2d25edd P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:28.674014 29036 catalog_manager.cc:5484] T d75abb6dc8a247d796194acad2d25edd P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:28.716413 29036 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:28.721153 29556 tablet_service.cc:1467] Processing CreateTablet for tablet b3d568eb61af4227b0574949cf181a46 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c229ae080e524715a7c3839b9eabd4ac]), partition=
I20240502 20:46:28.721426 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3d568eb61af4227b0574949cf181a46. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:28.721520 29425 tablet_service.cc:1467] Processing CreateTablet for tablet b3d568eb61af4227b0574949cf181a46 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c229ae080e524715a7c3839b9eabd4ac]), partition=
I20240502 20:46:28.721760 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3d568eb61af4227b0574949cf181a46. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:28.723621 31475 tablet_service.cc:1467] Processing CreateTablet for tablet b3d568eb61af4227b0574949cf181a46 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c229ae080e524715a7c3839b9eabd4ac]), partition=
I20240502 20:46:28.723802 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3d568eb61af4227b0574949cf181a46. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:28.723884 5478 tablet_bootstrap.cc:492] T b3d568eb61af4227b0574949cf181a46 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:28.723927 5479 tablet_bootstrap.cc:492] T b3d568eb61af4227b0574949cf181a46 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:28.724717 5478 tablet_bootstrap.cc:654] T b3d568eb61af4227b0574949cf181a46 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:28.724797 5479 tablet_bootstrap.cc:654] T b3d568eb61af4227b0574949cf181a46 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:28.725869 5477 tablet_bootstrap.cc:492] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:28.725980 5479 tablet_bootstrap.cc:492] T b3d568eb61af4227b0574949cf181a46 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:28.726065 5479 ts_tablet_manager.cc:1367] T b3d568eb61af4227b0574949cf181a46 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 20:46:28.726501 5479 raft_consensus.cc:348] T b3d568eb61af4227b0574949cf181a46 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:28.726611 5479 raft_consensus.cc:374] T b3d568eb61af4227b0574949cf181a46 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:28.726636 5479 raft_consensus.cc:729] T b3d568eb61af4227b0574949cf181a46 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:28.726747 5477 tablet_bootstrap.cc:654] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:28.726729 5479 consensus_queue.cc:260] T b3d568eb61af4227b0574949cf181a46 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:28.726985 5479 ts_tablet_manager.cc:1398] T b3d568eb61af4227b0574949cf181a46 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:28.728075 5478 tablet_bootstrap.cc:492] T b3d568eb61af4227b0574949cf181a46 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:28.728154 5478 ts_tablet_manager.cc:1367] T b3d568eb61af4227b0574949cf181a46 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:28.728571 5478 raft_consensus.cc:348] T b3d568eb61af4227b0574949cf181a46 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:28.728680 5478 raft_consensus.cc:374] T b3d568eb61af4227b0574949cf181a46 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:28.728709 5478 raft_consensus.cc:729] T b3d568eb61af4227b0574949cf181a46 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:28.728838 5478 consensus_queue.cc:260] T b3d568eb61af4227b0574949cf181a46 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:28.729128 5478 ts_tablet_manager.cc:1398] T b3d568eb61af4227b0574949cf181a46 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:28.729394 5477 tablet_bootstrap.cc:492] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:28.729466 5477 ts_tablet_manager.cc:1367] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:28.729882 5477 raft_consensus.cc:348] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:28.729993 5477 raft_consensus.cc:374] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:28.730033 5477 raft_consensus.cc:729] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:28.730154 5477 consensus_queue.cc:260] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:28.730429 5477 ts_tablet_manager.cc:1398] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:28.733466 5558 raft_consensus.cc:482] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:28.733559 5558 raft_consensus.cc:504] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:28.733875 5558 leader_election.cc:288] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:28.734108 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3d568eb61af4227b0574949cf181a46" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:28.734220 29577 raft_consensus.cc:2454] T b3d568eb61af4227b0574949cf181a46 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:28.734508 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3d568eb61af4227b0574949cf181a46" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:28.734598 31429 leader_election.cc:302] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:28.734635 29444 raft_consensus.cc:2454] T b3d568eb61af4227b0574949cf181a46 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:28.734918 5558 raft_consensus.cc:2781] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:28.734982 5558 raft_consensus.cc:482] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:28.735020 5558 raft_consensus.cc:3037] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:28.735888 5558 raft_consensus.cc:504] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:28.736202 5558 leader_election.cc:288] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:28.736449 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3d568eb61af4227b0574949cf181a46" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:28.736451 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3d568eb61af4227b0574949cf181a46" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:28.736546 29577 raft_consensus.cc:3037] T b3d568eb61af4227b0574949cf181a46 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:28.736559 29444 raft_consensus.cc:3037] T b3d568eb61af4227b0574949cf181a46 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:28.737332 29444 raft_consensus.cc:2454] T b3d568eb61af4227b0574949cf181a46 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:28.737337 29577 raft_consensus.cc:2454] T b3d568eb61af4227b0574949cf181a46 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:28.737555 31430 leader_election.cc:302] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:28.737761 5558 raft_consensus.cc:2781] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:28.737826 5558 raft_consensus.cc:686] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Becoming Leader. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Running, Role: LEADER
I20240502 20:46:28.737936 5558 consensus_queue.cc:237] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:28.739580 29036 catalog_manager.cc:5484] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69). New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:28.801005 4216 raft_consensus.cc:482] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:28.801110 4216 raft_consensus.cc:504] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:28.801573 4216 leader_election.cc:288] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:28.801734 5471 raft_consensus.cc:482] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:28.801822 5471 raft_consensus.cc:504] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:28.801826 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d60f6c42e446a1bc2bc9ea4c7e55b8" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:28.801962 31888 raft_consensus.cc:2454] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:28.801968 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d60f6c42e446a1bc2bc9ea4c7e55b8" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:28.802094 29444 raft_consensus.cc:2454] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:28.802187 29511 leader_election.cc:302] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:28.802234 5471 leader_election.cc:288] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:28.802345 4216 raft_consensus.cc:2781] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:28.802397 4216 raft_consensus.cc:482] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:28.802433 4216 raft_consensus.cc:3037] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:28.802569 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d60f6c42e446a1bc2bc9ea4c7e55b8" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:28.802848 31887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d60f6c42e446a1bc2bc9ea4c7e55b8" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:28.802956 31887 raft_consensus.cc:2454] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:28.803197 29380 leader_election.cc:302] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dbcdc6ac635b44b185c3eba63f66e9d6, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:28.803357 5471 raft_consensus.cc:2781] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:28.803418 5471 raft_consensus.cc:482] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:28.803457 5471 raft_consensus.cc:3037] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:28.803536 4216 raft_consensus.cc:504] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:28.803884 4216 leader_election.cc:288] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Requested vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:28.804111 29577 raft_consensus.cc:2379] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbcdc6ac635b44b185c3eba63f66e9d6 in current term 1: Already voted for candidate db17f26a73f3447b92aa9a947e3a7057 in this term.
I20240502 20:46:28.804140 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d60f6c42e446a1bc2bc9ea4c7e55b8" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:28.804240 31887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d60f6c42e446a1bc2bc9ea4c7e55b8" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:28.804337 31887 raft_consensus.cc:3037] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:28.804461 5471 raft_consensus.cc:504] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:28.804638 29444 raft_consensus.cc:2379] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate db17f26a73f3447b92aa9a947e3a7057 in current term 1: Already voted for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in this term.
I20240502 20:46:28.804771 5471 leader_election.cc:288] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:28.805361 31887 raft_consensus.cc:2454] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:28.805424 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d60f6c42e446a1bc2bc9ea4c7e55b8" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:28.805547 29577 raft_consensus.cc:2379] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate dbcdc6ac635b44b185c3eba63f66e9d6 in current term 1: Already voted for candidate db17f26a73f3447b92aa9a947e3a7057 in this term.
I20240502 20:46:28.805619 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d60f6c42e446a1bc2bc9ea4c7e55b8" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:28.805756 31888 raft_consensus.cc:2379] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate dbcdc6ac635b44b185c3eba63f66e9d6 in current term 1: Already voted for candidate db17f26a73f3447b92aa9a947e3a7057 in this term.
I20240502 20:46:28.805925 29511 leader_election.cc:302] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, e6b60803d0764d228c863e2da65e3691; no voters: dbcdc6ac635b44b185c3eba63f66e9d6
I20240502 20:46:28.805943 29380 leader_election.cc:302] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbcdc6ac635b44b185c3eba63f66e9d6; no voters: db17f26a73f3447b92aa9a947e3a7057, e6b60803d0764d228c863e2da65e3691
I20240502 20:46:28.806105 4216 raft_consensus.cc:2781] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:28.806123 5471 raft_consensus.cc:2726] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240502 20:46:28.806185 4216 raft_consensus.cc:686] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Becoming Leader. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Running, Role: LEADER
I20240502 20:46:28.806308 4216 consensus_queue.cc:237] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:28.808347 29036 catalog_manager.cc:5484] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: term changed from 0 to 1, leader changed from <none> to db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:28.829185 29295 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:28.833357 29556 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:28.839388 29162 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:28.840610 31868 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:28.847088 29036 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:28.849248 29425 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:28.852164 29574 raft_consensus.cc:1261] T b3d568eb61af4227b0574949cf181a46 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:28.852597 4619 consensus_queue.cc:1035] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:28.853046 29444 raft_consensus.cc:1261] T b3d568eb61af4227b0574949cf181a46 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:28.853384 4619 consensus_queue.cc:1035] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:28.856647 29036 catalog_manager.cc:5576] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897) does not have the latest schema for tablet b3d568eb61af4227b0574949cf181a46 (table rebalancer_dynamic_table [id=c229ae080e524715a7c3839b9eabd4ac]). Expected version 1 got 0
I20240502 20:46:28.859730 29036 catalog_manager.cc:5576] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095) does not have the latest schema for tablet b3d568eb61af4227b0574949cf181a46 (table rebalancer_dynamic_table [id=c229ae080e524715a7c3839b9eabd4ac]). Expected version 1 got 0
I20240502 20:46:28.863792 31605 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:28.867974 29036 catalog_manager.cc:5576] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897) does not have the latest schema for tablet b3d568eb61af4227b0574949cf181a46 (table rebalancer_dynamic_table [id=c229ae080e524715a7c3839b9eabd4ac]). Expected version 1 got 0
I20240502 20:46:28.868542 5579 tablet.cc:1722] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:28.868907 5579 tablet.cc:1620] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:28.874863 31735 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:28.878607 5580 tablet.cc:1722] T b3d568eb61af4227b0574949cf181a46 P db17f26a73f3447b92aa9a947e3a7057: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:28.878963 5580 tablet.cc:1620] T b3d568eb61af4227b0574949cf181a46 P db17f26a73f3447b92aa9a947e3a7057: MemRowSet was empty: no flush needed.
I20240502 20:46:28.888639 29036 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=c229ae080e524715a7c3839b9eabd4ac] alter complete (version 1)
I20240502 20:46:28.896247 31475 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:28.905357 5427 tablet.cc:1722] T b3d568eb61af4227b0574949cf181a46 P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:28.905670 5427 tablet.cc:1620] T b3d568eb61af4227b0574949cf181a46 P dbcdc6ac635b44b185c3eba63f66e9d6: MemRowSet was empty: no flush needed.
I20240502 20:46:28.906757 29036 catalog_manager.cc:5576] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095) does not have the latest schema for tablet b3d568eb61af4227b0574949cf181a46 (table rebalancer_dynamic_table [id=c229ae080e524715a7c3839b9eabd4ac]). Expected version 1 got 0
I20240502 20:46:28.940814 29574 consensus_queue.cc:237] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:28.942219 31755 raft_consensus.cc:1261] T 28d6a4d9c92141f794d6ee04335cf9b8 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:28.942739 5346 consensus_queue.cc:1035] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:28.942781 31888 raft_consensus.cc:1261] T 28d6a4d9c92141f794d6ee04335cf9b8 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:28.943094 5346 consensus_queue.cc:1035] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:28.944360 4216 raft_consensus.cc:2932] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:28.945726 31755 raft_consensus.cc:2932] T 28d6a4d9c92141f794d6ee04335cf9b8 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:28.947173 29036 catalog_manager.cc:5484] T 28d6a4d9c92141f794d6ee04335cf9b8 P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: config changed from index 2 to 3, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) evicted. New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:28.948562 31888 raft_consensus.cc:2932] T 28d6a4d9c92141f794d6ee04335cf9b8 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:28.951179 31475 tablet_service.cc:1514] Processing DeleteTablet for tablet 28d6a4d9c92141f794d6ee04335cf9b8 with delete_type TABLET_DATA_TOMBSTONED (TS 2a9c1af8cf814030bde56b67ca5c2fbe not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:40276
I20240502 20:46:28.951782 5603 tablet_replica.cc:330] T 28d6a4d9c92141f794d6ee04335cf9b8 P 2a9c1af8cf814030bde56b67ca5c2fbe: stopping tablet replica
I20240502 20:46:28.951982 5603 raft_consensus.cc:2229] T 28d6a4d9c92141f794d6ee04335cf9b8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:28.952157 5603 raft_consensus.cc:2258] T 28d6a4d9c92141f794d6ee04335cf9b8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:28.952713 5603 ts_tablet_manager.cc:1875] T 28d6a4d9c92141f794d6ee04335cf9b8 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:28.955058 5603 ts_tablet_manager.cc:1888] T 28d6a4d9c92141f794d6ee04335cf9b8 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 20:46:28.955135 5603 log.cc:1196] T 28d6a4d9c92141f794d6ee04335cf9b8 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/wals/28d6a4d9c92141f794d6ee04335cf9b8
I20240502 20:46:28.955754 29028 catalog_manager.cc:4830] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): tablet 28d6a4d9c92141f794d6ee04335cf9b8 (table rebalancer_extra_table_4 [id=57203d81e8ab41bcb24361b9b6107024]) successfully deleted
I20240502 20:46:28.990736 31495 consensus_queue.cc:237] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:28.992147 29315 raft_consensus.cc:1261] T 0922994ff62d4e648d5132cb08b6a3c8 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:28.992240 31888 raft_consensus.cc:1261] T 0922994ff62d4e648d5132cb08b6a3c8 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 20:46:28.992559 31430 consensus_peers.cc:485] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): Couldn't send request to peer dbcdc6ac635b44b185c3eba63f66e9d6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0922994ff62d4e648d5132cb08b6a3c8. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:28.992633 4975 consensus_queue.cc:1035] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:28.992954 4619 consensus_queue.cc:1035] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:28.995384 4975 raft_consensus.cc:2932] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:28.995903 31887 raft_consensus.cc:2932] T 0922994ff62d4e648d5132cb08b6a3c8 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:28.997718 29315 raft_consensus.cc:2932] T 0922994ff62d4e648d5132cb08b6a3c8 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:28.999325 29041 catalog_manager.cc:5484] T 0922994ff62d4e648d5132cb08b6a3c8 P e6b60803d0764d228c863e2da65e3691 reported cstate change: config changed from index -1 to 2, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) added. New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:28.999850 29038 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:29.006726 5579 tablet.cc:1722] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:29.007071 5579 tablet.cc:1620] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:29.007938 5580 tablet.cc:1722] T b3d568eb61af4227b0574949cf181a46 P db17f26a73f3447b92aa9a947e3a7057: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:29.008211 5580 tablet.cc:1620] T b3d568eb61af4227b0574949cf181a46 P db17f26a73f3447b92aa9a947e3a7057: MemRowSet was empty: no flush needed.
I20240502 20:46:29.009253 5427 tablet.cc:1722] T b3d568eb61af4227b0574949cf181a46 P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:29.009564 5427 tablet.cc:1620] T b3d568eb61af4227b0574949cf181a46 P dbcdc6ac635b44b185c3eba63f66e9d6: MemRowSet was empty: no flush needed.
I20240502 20:46:29.013370 29042 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=c229ae080e524715a7c3839b9eabd4ac] alter complete (version 2)
I20240502 20:46:29.037540 29175 consensus_queue.cc:237] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:29.039088 29446 raft_consensus.cc:1261] T 3943d49acd0341c1a923d900281f1591 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:29.039175 29315 raft_consensus.cc:1261] T 3943d49acd0341c1a923d900281f1591 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:29.039551 5656 consensus_queue.cc:1035] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 20:46:29.039759 29118 consensus_peers.cc:485] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d -> Peer e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): Couldn't send request to peer e6b60803d0764d228c863e2da65e3691. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3943d49acd0341c1a923d900281f1591. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:29.039785 5655 consensus_queue.cc:1035] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:29.041452 5032 raft_consensus.cc:2932] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:29.041847 29446 raft_consensus.cc:2932] T 3943d49acd0341c1a923d900281f1591 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:29.044595 29036 catalog_manager.cc:5484] T 3943d49acd0341c1a923d900281f1591 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:29.045089 29315 raft_consensus.cc:2932] T 3943d49acd0341c1a923d900281f1591 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:29.056175 3289 consensus_queue.cc:1035] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:29.060253 5672 consensus_queue.cc:1035] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:29.066924 29574 consensus_queue.cc:237] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:29.068393 29446 raft_consensus.cc:1261] T e7642b18bcbb467187fc8c8b924b79df P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:29.068593 29315 raft_consensus.cc:1261] T e7642b18bcbb467187fc8c8b924b79df P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:29.068702 5656 consensus_queue.cc:1035] T d75abb6dc8a247d796194acad2d25edd P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 20:46:29.069007 29512 consensus_peers.cc:485] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 -> Peer 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): Couldn't send request to peer 2a9c1af8cf814030bde56b67ca5c2fbe. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7642b18bcbb467187fc8c8b924b79df. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:29.069216 5505 consensus_queue.cc:1035] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:29.069640 4216 consensus_queue.cc:1035] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:29.072132 5655 consensus_queue.cc:1035] T d75abb6dc8a247d796194acad2d25edd P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:29.072047 4216 raft_consensus.cc:2932] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:29.072733 29446 raft_consensus.cc:2932] T e7642b18bcbb467187fc8c8b924b79df P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:29.074254 29315 raft_consensus.cc:2932] T e7642b18bcbb467187fc8c8b924b79df P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:29.075480 29036 catalog_manager.cc:5484] T e7642b18bcbb467187fc8c8b924b79df P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index -1 to 4, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:29.117556 29042 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:29.124189 5579 tablet.cc:1722] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:29.124955 5580 tablet.cc:1722] T b3d568eb61af4227b0574949cf181a46 P db17f26a73f3447b92aa9a947e3a7057: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:29.127203 5427 tablet.cc:1722] T b3d568eb61af4227b0574949cf181a46 P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:29.133193 29040 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=c229ae080e524715a7c3839b9eabd4ac] alter complete (version 3)
I20240502 20:46:29.228468 5505 consensus_queue.cc:1035] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:29.231639 4216 consensus_queue.cc:1035] T 28d60f6c42e446a1bc2bc9ea4c7e55b8 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:29.236840 29034 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:29.236959 29034 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:29.239853 29034 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet b3d568eb61af4227b0574949cf181a46
I20240502 20:46:29.240751 29556 tablet_service.cc:1514] Processing DeleteTablet for tablet b3d568eb61af4227b0574949cf181a46 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:29 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:29.240999 31475 tablet_service.cc:1514] Processing DeleteTablet for tablet b3d568eb61af4227b0574949cf181a46 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:29 UTC) from {username='slave'} at 127.0.0.1:40276
I20240502 20:46:29.241179 29425 tablet_service.cc:1514] Processing DeleteTablet for tablet b3d568eb61af4227b0574949cf181a46 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:29 UTC) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:29.241541 5603 tablet_replica.cc:330] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe: stopping tablet replica
I20240502 20:46:29.241696 5603 raft_consensus.cc:2229] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:29.241906 5603 raft_consensus.cc:2258] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:29.243047 5603 ts_tablet_manager.cc:1875] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:29.245358 5603 ts_tablet_manager.cc:1888] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:29.245441 5603 log.cc:1196] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/wals/b3d568eb61af4227b0574949cf181a46
I20240502 20:46:29.246712 5603 ts_tablet_manager.cc:1909] T b3d568eb61af4227b0574949cf181a46 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting consensus metadata
I20240502 20:46:29.247498 29028 catalog_manager.cc:4830] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): tablet b3d568eb61af4227b0574949cf181a46 (table rebalancer_dynamic_table_new_name [id=c229ae080e524715a7c3839b9eabd4ac]) successfully deleted
I20240502 20:46:29.247848 5779 tablet_replica.cc:330] T b3d568eb61af4227b0574949cf181a46 P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:29.248042 5781 tablet_replica.cc:330] T b3d568eb61af4227b0574949cf181a46 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:29.248458 5779 raft_consensus.cc:2229] T b3d568eb61af4227b0574949cf181a46 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:29.248656 5781 raft_consensus.cc:2229] T b3d568eb61af4227b0574949cf181a46 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:29.248850 5781 raft_consensus.cc:2258] T b3d568eb61af4227b0574949cf181a46 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:29.249253 5779 raft_consensus.cc:2258] T b3d568eb61af4227b0574949cf181a46 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:29.250142 5781 ts_tablet_manager.cc:1875] T b3d568eb61af4227b0574949cf181a46 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:29.251457 5779 ts_tablet_manager.cc:1875] T b3d568eb61af4227b0574949cf181a46 P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:29.252897 5781 ts_tablet_manager.cc:1888] T b3d568eb61af4227b0574949cf181a46 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:29.252995 29162 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:29.253022 5781 log.cc:1196] T b3d568eb61af4227b0574949cf181a46 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/b3d568eb61af4227b0574949cf181a46
I20240502 20:46:29.253310 5781 ts_tablet_manager.cc:1909] T b3d568eb61af4227b0574949cf181a46 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting consensus metadata
I20240502 20:46:29.254315 29295 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:29.255378 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet b3d568eb61af4227b0574949cf181a46 (table rebalancer_dynamic_table_new_name [id=c229ae080e524715a7c3839b9eabd4ac]) successfully deleted
I20240502 20:46:29.259373 29556 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:29.266188 31735 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:29.270150 31475 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:29.276067 5779 ts_tablet_manager.cc:1888] T b3d568eb61af4227b0574949cf181a46 P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:29.276265 5779 log.cc:1196] T b3d568eb61af4227b0574949cf181a46 P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/b3d568eb61af4227b0574949cf181a46
I20240502 20:46:29.276644 5779 ts_tablet_manager.cc:1909] T b3d568eb61af4227b0574949cf181a46 P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:29.277333 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet b3d568eb61af4227b0574949cf181a46 (table rebalancer_dynamic_table_new_name [id=c229ae080e524715a7c3839b9eabd4ac]) successfully deleted
I20240502 20:46:29.281806 29425 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:29.288558 31868 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:29.310395 31605 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:29.320996 29034 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_44"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:29.327796 31735 tablet_service.cc:1467] Processing CreateTablet for tablet 0b9f8b50d5df4c57b5bf058427befec1 (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=1d2cb2e930ad4d819fb2959ed78d9a3c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:29.328101 31735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b9f8b50d5df4c57b5bf058427befec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:29.328114 29162 tablet_service.cc:1467] Processing CreateTablet for tablet 0b9f8b50d5df4c57b5bf058427befec1 (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=1d2cb2e930ad4d819fb2959ed78d9a3c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:29.328377 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b9f8b50d5df4c57b5bf058427befec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:29.329120 31475 tablet_service.cc:1467] Processing CreateTablet for tablet 0b9f8b50d5df4c57b5bf058427befec1 (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=1d2cb2e930ad4d819fb2959ed78d9a3c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:29.329123 29425 tablet_service.cc:1467] Processing CreateTablet for tablet 653cbec396434b4dac0a1928b21d0ebd (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=1d2cb2e930ad4d819fb2959ed78d9a3c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:29.329375 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b9f8b50d5df4c57b5bf058427befec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:29.329382 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 653cbec396434b4dac0a1928b21d0ebd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:29.330591 29295 tablet_service.cc:1467] Processing CreateTablet for tablet d670c6c209764e64b0ba7373dfc1f572 (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=1d2cb2e930ad4d819fb2959ed78d9a3c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:29.330741 31474 tablet_service.cc:1467] Processing CreateTablet for tablet d670c6c209764e64b0ba7373dfc1f572 (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=1d2cb2e930ad4d819fb2959ed78d9a3c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:29.330837 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d670c6c209764e64b0ba7373dfc1f572. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:29.330950 31474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d670c6c209764e64b0ba7373dfc1f572. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:29.332657 31605 tablet_service.cc:1467] Processing CreateTablet for tablet 653cbec396434b4dac0a1928b21d0ebd (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=1d2cb2e930ad4d819fb2959ed78d9a3c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:29.332902 31605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 653cbec396434b4dac0a1928b21d0ebd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:29.333068 31868 tablet_service.cc:1467] Processing CreateTablet for tablet 653cbec396434b4dac0a1928b21d0ebd (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=1d2cb2e930ad4d819fb2959ed78d9a3c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:29.333323 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 653cbec396434b4dac0a1928b21d0ebd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:29.333561 31735 tablet_service.cc:1467] Processing CreateTablet for tablet d670c6c209764e64b0ba7373dfc1f572 (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=1d2cb2e930ad4d819fb2959ed78d9a3c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:29.333765 31735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d670c6c209764e64b0ba7373dfc1f572. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:29.335410 5784 tablet_bootstrap.cc:492] T 653cbec396434b4dac0a1928b21d0ebd P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:29.336418 5786 tablet_bootstrap.cc:492] T d670c6c209764e64b0ba7373dfc1f572 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:29.336769 5785 tablet_bootstrap.cc:492] T 0b9f8b50d5df4c57b5bf058427befec1 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:29.337318 5786 tablet_bootstrap.cc:654] T d670c6c209764e64b0ba7373dfc1f572 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:29.337630 5785 tablet_bootstrap.cc:654] T 0b9f8b50d5df4c57b5bf058427befec1 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:29.337929 5787 tablet_bootstrap.cc:492] T 653cbec396434b4dac0a1928b21d0ebd P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:29.338299 5784 tablet_bootstrap.cc:654] T 653cbec396434b4dac0a1928b21d0ebd P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:29.338811 5787 tablet_bootstrap.cc:654] T 653cbec396434b4dac0a1928b21d0ebd P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:29.339181 5782 tablet_bootstrap.cc:492] T 0b9f8b50d5df4c57b5bf058427befec1 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:29.339270 5784 tablet_bootstrap.cc:492] T 653cbec396434b4dac0a1928b21d0ebd P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:29.339342 5784 ts_tablet_manager.cc:1367] T 653cbec396434b4dac0a1928b21d0ebd P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:29.339741 5784 raft_consensus.cc:348] T 653cbec396434b4dac0a1928b21d0ebd P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:29.339833 5784 raft_consensus.cc:374] T 653cbec396434b4dac0a1928b21d0ebd P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:29.339857 5784 raft_consensus.cc:729] T 653cbec396434b4dac0a1928b21d0ebd P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:29.339953 5784 consensus_queue.cc:260] T 653cbec396434b4dac0a1928b21d0ebd P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:29.340193 5784 ts_tablet_manager.cc:1398] T 653cbec396434b4dac0a1928b21d0ebd P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:29.340607 5782 tablet_bootstrap.cc:654] T 0b9f8b50d5df4c57b5bf058427befec1 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:29.341434 5791 tablet_bootstrap.cc:492] T 653cbec396434b4dac0a1928b21d0ebd P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:29.342231 5782 tablet_bootstrap.cc:492] T 0b9f8b50d5df4c57b5bf058427befec1 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:29.342314 5782 ts_tablet_manager.cc:1367] T 0b9f8b50d5df4c57b5bf058427befec1 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:29.342334 5783 tablet_bootstrap.cc:492] T 0b9f8b50d5df4c57b5bf058427befec1 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:29.342741 5782 raft_consensus.cc:348] T 0b9f8b50d5df4c57b5bf058427befec1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:29.342844 5782 raft_consensus.cc:374] T 0b9f8b50d5df4c57b5bf058427befec1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:29.342885 5782 raft_consensus.cc:729] T 0b9f8b50d5df4c57b5bf058427befec1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:29.343006 5782 consensus_queue.cc:260] T 0b9f8b50d5df4c57b5bf058427befec1 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:29.343282 5782 ts_tablet_manager.cc:1398] T 0b9f8b50d5df4c57b5bf058427befec1 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:29.343533 5791 tablet_bootstrap.cc:654] T 653cbec396434b4dac0a1928b21d0ebd P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:29.344578 5782 tablet_bootstrap.cc:492] T d670c6c209764e64b0ba7373dfc1f572 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:29.345415 5782 tablet_bootstrap.cc:654] T d670c6c209764e64b0ba7373dfc1f572 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:29.346387 5782 tablet_bootstrap.cc:492] T d670c6c209764e64b0ba7373dfc1f572 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:29.346509 5782 ts_tablet_manager.cc:1367] T d670c6c209764e64b0ba7373dfc1f572 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:29.347474 5782 raft_consensus.cc:348] T d670c6c209764e64b0ba7373dfc1f572 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:29.347599 5782 raft_consensus.cc:374] T d670c6c209764e64b0ba7373dfc1f572 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:29.347640 5782 raft_consensus.cc:729] T d670c6c209764e64b0ba7373dfc1f572 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:29.347767 5782 consensus_queue.cc:260] T d670c6c209764e64b0ba7373dfc1f572 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:29.348331 5782 ts_tablet_manager.cc:1398] T d670c6c209764e64b0ba7373dfc1f572 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:29.351538 29034 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:29.355686 5783 tablet_bootstrap.cc:654] T 0b9f8b50d5df4c57b5bf058427befec1 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:29.357518 5787 tablet_bootstrap.cc:492] T 653cbec396434b4dac0a1928b21d0ebd P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:29.357617 5787 ts_tablet_manager.cc:1367] T 653cbec396434b4dac0a1928b21d0ebd P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20240502 20:46:29.357714 5783 tablet_bootstrap.cc:492] T 0b9f8b50d5df4c57b5bf058427befec1 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:29.357789 5783 ts_tablet_manager.cc:1367] T 0b9f8b50d5df4c57b5bf058427befec1 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240502 20:46:29.357828 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 04ebdefddc1349c586b5242a6501661d (DEFAULT_TABLE table=rebalancer_dynamic_table [id=34c07c8d7d144b81946dee5ca2537aa7]), partition=
I20240502 20:46:29.358065 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04ebdefddc1349c586b5242a6501661d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:29.358064 5787 raft_consensus.cc:348] T 653cbec396434b4dac0a1928b21d0ebd P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:29.358162 5787 raft_consensus.cc:374] T 653cbec396434b4dac0a1928b21d0ebd P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:29.358215 5787 raft_consensus.cc:729] T 653cbec396434b4dac0a1928b21d0ebd P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:29.358167 5783 raft_consensus.cc:348] T 0b9f8b50d5df4c57b5bf058427befec1 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:29.358256 5783 raft_consensus.cc:374] T 0b9f8b50d5df4c57b5bf058427befec1 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:29.358284 5783 raft_consensus.cc:729] T 0b9f8b50d5df4c57b5bf058427befec1 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:29.358328 5787 consensus_queue.cc:260] T 653cbec396434b4dac0a1928b21d0ebd P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:29.358409 5783 consensus_queue.cc:260] T 0b9f8b50d5df4c57b5bf058427befec1 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:29.358639 5787 ts_tablet_manager.cc:1398] T 653cbec396434b4dac0a1928b21d0ebd P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:29.358930 29162 tablet_service.cc:1467] Processing CreateTablet for tablet 04ebdefddc1349c586b5242a6501661d (DEFAULT_TABLE table=rebalancer_dynamic_table [id=34c07c8d7d144b81946dee5ca2537aa7]), partition=
I20240502 20:46:29.358964 31605 tablet_service.cc:1467] Processing CreateTablet for tablet 04ebdefddc1349c586b5242a6501661d (DEFAULT_TABLE table=rebalancer_dynamic_table [id=34c07c8d7d144b81946dee5ca2537aa7]), partition=
I20240502 20:46:29.359130 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04ebdefddc1349c586b5242a6501661d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:29.359133 31605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04ebdefddc1349c586b5242a6501661d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:29.361097 5783 ts_tablet_manager.cc:1398] T 0b9f8b50d5df4c57b5bf058427befec1 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:29.361258 5783 tablet_bootstrap.cc:492] T 04ebdefddc1349c586b5242a6501661d P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:29.362064 5783 tablet_bootstrap.cc:654] T 04ebdefddc1349c586b5242a6501661d P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:29.363150 5783 tablet_bootstrap.cc:492] T 04ebdefddc1349c586b5242a6501661d P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:29.363229 5783 ts_tablet_manager.cc:1367] T 04ebdefddc1349c586b5242a6501661d P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:29.363662 5783 raft_consensus.cc:348] T 04ebdefddc1349c586b5242a6501661d P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:29.363760 5783 raft_consensus.cc:374] T 04ebdefddc1349c586b5242a6501661d P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:29.363791 5783 raft_consensus.cc:729] T 04ebdefddc1349c586b5242a6501661d P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:29.363912 5783 consensus_queue.cc:260] T 04ebdefddc1349c586b5242a6501661d P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:29.364182 5783 ts_tablet_manager.cc:1398] T 04ebdefddc1349c586b5242a6501661d P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:29.368023 5786 tablet_bootstrap.cc:492] T d670c6c209764e64b0ba7373dfc1f572 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:29.368111 5786 ts_tablet_manager.cc:1367] T d670c6c209764e64b0ba7373dfc1f572 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.032s user 0.001s sys 0.000s
I20240502 20:46:29.368556 5786 raft_consensus.cc:348] T d670c6c209764e64b0ba7373dfc1f572 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:29.368659 5786 raft_consensus.cc:374] T d670c6c209764e64b0ba7373dfc1f572 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:29.368692 5786 raft_consensus.cc:729] T d670c6c209764e64b0ba7373dfc1f572 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:29.368779 5786 consensus_queue.cc:260] T d670c6c209764e64b0ba7373dfc1f572 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:29.369037 5786 ts_tablet_manager.cc:1398] T d670c6c209764e64b0ba7373dfc1f572 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:29.369201 5786 tablet_bootstrap.cc:492] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:29.369948 5786 tablet_bootstrap.cc:654] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:29.370847 5786 tablet_bootstrap.cc:492] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:29.370926 5786 ts_tablet_manager.cc:1367] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:29.371047 5785 tablet_bootstrap.cc:492] T 0b9f8b50d5df4c57b5bf058427befec1 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:29.371150 5785 ts_tablet_manager.cc:1367] T 0b9f8b50d5df4c57b5bf058427befec1 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.034s user 0.002s sys 0.000s
I20240502 20:46:29.371304 5786 raft_consensus.cc:348] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:29.371384 5786 raft_consensus.cc:374] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:29.371412 5786 raft_consensus.cc:729] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:29.371584 5785 raft_consensus.cc:348] T 0b9f8b50d5df4c57b5bf058427befec1 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:29.371692 5785 raft_consensus.cc:374] T 0b9f8b50d5df4c57b5bf058427befec1 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:29.371726 5785 raft_consensus.cc:729] T 0b9f8b50d5df4c57b5bf058427befec1 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:29.371835 5785 consensus_queue.cc:260] T 0b9f8b50d5df4c57b5bf058427befec1 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:29.372112 5785 ts_tablet_manager.cc:1398] T 0b9f8b50d5df4c57b5bf058427befec1 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:29.372308 5785 tablet_bootstrap.cc:492] T d670c6c209764e64b0ba7373dfc1f572 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:29.373214 5785 tablet_bootstrap.cc:654] T d670c6c209764e64b0ba7373dfc1f572 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:29.372867 5786 consensus_queue.cc:260] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:29.374703 5786 ts_tablet_manager.cc:1398] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:29.376068 5785 tablet_bootstrap.cc:492] T d670c6c209764e64b0ba7373dfc1f572 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:29.376140 5785 ts_tablet_manager.cc:1367] T d670c6c209764e64b0ba7373dfc1f572 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:29.376493 5785 raft_consensus.cc:348] T d670c6c209764e64b0ba7373dfc1f572 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:29.376575 5785 raft_consensus.cc:374] T d670c6c209764e64b0ba7373dfc1f572 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:29.376617 5785 raft_consensus.cc:729] T d670c6c209764e64b0ba7373dfc1f572 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:29.376713 5785 consensus_queue.cc:260] T d670c6c209764e64b0ba7373dfc1f572 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:29.376935 5785 ts_tablet_manager.cc:1398] T d670c6c209764e64b0ba7373dfc1f572 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:29.381111 5791 tablet_bootstrap.cc:492] T 653cbec396434b4dac0a1928b21d0ebd P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:29.381209 5791 ts_tablet_manager.cc:1367] T 653cbec396434b4dac0a1928b21d0ebd P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.040s user 0.002s sys 0.000s
I20240502 20:46:29.381615 5791 raft_consensus.cc:348] T 653cbec396434b4dac0a1928b21d0ebd P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:29.381723 5791 raft_consensus.cc:374] T 653cbec396434b4dac0a1928b21d0ebd P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:29.381753 5791 raft_consensus.cc:729] T 653cbec396434b4dac0a1928b21d0ebd P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:29.381851 5791 consensus_queue.cc:260] T 653cbec396434b4dac0a1928b21d0ebd P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:29.382093 5791 ts_tablet_manager.cc:1398] T 653cbec396434b4dac0a1928b21d0ebd P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:29.382234 5791 tablet_bootstrap.cc:492] T 04ebdefddc1349c586b5242a6501661d P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:29.383051 5791 tablet_bootstrap.cc:654] T 04ebdefddc1349c586b5242a6501661d P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:29.385169 5791 tablet_bootstrap.cc:492] T 04ebdefddc1349c586b5242a6501661d P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:29.385254 5791 ts_tablet_manager.cc:1367] T 04ebdefddc1349c586b5242a6501661d P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:29.385706 5791 raft_consensus.cc:348] T 04ebdefddc1349c586b5242a6501661d P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:29.385818 5791 raft_consensus.cc:374] T 04ebdefddc1349c586b5242a6501661d P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:29.385849 5791 raft_consensus.cc:729] T 04ebdefddc1349c586b5242a6501661d P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:29.385968 5791 consensus_queue.cc:260] T 04ebdefddc1349c586b5242a6501661d P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:29.386229 5791 ts_tablet_manager.cc:1398] T 04ebdefddc1349c586b5242a6501661d P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:29.520282 4444 raft_consensus.cc:482] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:29.520401 4444 raft_consensus.cc:504] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:29.520828 4444 leader_election.cc:288] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:29.521247 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04ebdefddc1349c586b5242a6501661d" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:29.521406 29182 raft_consensus.cc:2454] T 04ebdefddc1349c586b5242a6501661d P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:29.521688 29251 leader_election.cc:302] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:29.521808 31624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04ebdefddc1349c586b5242a6501661d" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:29.521888 4444 raft_consensus.cc:2781] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:29.521948 31624 raft_consensus.cc:2454] T 04ebdefddc1349c586b5242a6501661d P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:29.521945 4444 raft_consensus.cc:482] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:29.522009 4444 raft_consensus.cc:3037] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:29.523078 4444 raft_consensus.cc:504] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:29.523401 4444 leader_election.cc:288] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:29.523694 31624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04ebdefddc1349c586b5242a6501661d" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:29.523798 31624 raft_consensus.cc:3037] T 04ebdefddc1349c586b5242a6501661d P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:29.523773 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04ebdefddc1349c586b5242a6501661d" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:29.523881 29182 raft_consensus.cc:3037] T 04ebdefddc1349c586b5242a6501661d P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:29.524780 31624 raft_consensus.cc:2454] T 04ebdefddc1349c586b5242a6501661d P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:29.524797 29182 raft_consensus.cc:2454] T 04ebdefddc1349c586b5242a6501661d P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:29.525075 29249 leader_election.cc:302] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 315915efcf2b413ca7f9d874338e1b2d; no voters:
I20240502 20:46:29.525307 4444 raft_consensus.cc:2781] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:29.525475 4444 raft_consensus.cc:686] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:29.525580 4444 consensus_queue.cc:237] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:29.527611 29041 catalog_manager.cc:5484] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:29.551527 5845 ts_tablet_manager.cc:902] T 0922994ff62d4e648d5132cb08b6a3c8 P dbcdc6ac635b44b185c3eba63f66e9d6: Initiating tablet copy from peer 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:29.551978 5845 tablet_copy_client.cc:308] T 0922994ff62d4e648d5132cb08b6a3c8 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.69:43887
I20240502 20:46:29.552385 31515 tablet_copy_service.cc:140] P 2a9c1af8cf814030bde56b67ca5c2fbe: Received BeginTabletCopySession request for tablet 0922994ff62d4e648d5132cb08b6a3c8 from peer dbcdc6ac635b44b185c3eba63f66e9d6 ({username='slave'} at 127.22.45.67:46915)
I20240502 20:46:29.552503 31515 tablet_copy_service.cc:161] P 2a9c1af8cf814030bde56b67ca5c2fbe: Beginning new tablet copy session on tablet 0922994ff62d4e648d5132cb08b6a3c8 from peer dbcdc6ac635b44b185c3eba63f66e9d6 at {username='slave'} at 127.22.45.67:46915: session id = dbcdc6ac635b44b185c3eba63f66e9d6-0922994ff62d4e648d5132cb08b6a3c8
I20240502 20:46:29.553470 31515 tablet_copy_source_session.cc:205] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:29.554316 5845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0922994ff62d4e648d5132cb08b6a3c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:29.556469 5845 tablet_copy_client.cc:783] T 0922994ff62d4e648d5132cb08b6a3c8 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:29.556614 5845 tablet_copy_client.cc:647] T 0922994ff62d4e648d5132cb08b6a3c8 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:29.557540 5845 tablet_copy_client.cc:523] T 0922994ff62d4e648d5132cb08b6a3c8 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:29.558434 5845 tablet_bootstrap.cc:492] T 0922994ff62d4e648d5132cb08b6a3c8 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:29.561273 5845 tablet_bootstrap.cc:492] T 0922994ff62d4e648d5132cb08b6a3c8 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:29.561482 5845 tablet_bootstrap.cc:492] T 0922994ff62d4e648d5132cb08b6a3c8 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap complete.
I20240502 20:46:29.561625 5845 ts_tablet_manager.cc:1367] T 0922994ff62d4e648d5132cb08b6a3c8 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20240502 20:46:29.561863 4619 raft_consensus.cc:482] T d670c6c209764e64b0ba7373dfc1f572 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:29.561954 4619 raft_consensus.cc:504] T d670c6c209764e64b0ba7373dfc1f572 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:29.562050 5845 raft_consensus.cc:348] T 0922994ff62d4e648d5132cb08b6a3c8 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:29.562144 5845 raft_consensus.cc:729] T 0922994ff62d4e648d5132cb08b6a3c8 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:29.562265 5845 consensus_queue.cc:260] T 0922994ff62d4e648d5132cb08b6a3c8 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:29.562427 4619 leader_election.cc:288] T d670c6c209764e64b0ba7373dfc1f572 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:29.562592 5845 ts_tablet_manager.cc:1398] T 0922994ff62d4e648d5132cb08b6a3c8 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 20:46:29.562855 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d670c6c209764e64b0ba7373dfc1f572" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:29.562985 31755 raft_consensus.cc:2454] T d670c6c209764e64b0ba7373dfc1f572 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:29.562989 31515 tablet_copy_service.cc:342] P 2a9c1af8cf814030bde56b67ca5c2fbe: Request end of tablet copy session dbcdc6ac635b44b185c3eba63f66e9d6-0922994ff62d4e648d5132cb08b6a3c8 received from {username='slave'} at 127.22.45.67:46915
I20240502 20:46:29.563052 31515 tablet_copy_service.cc:433] P 2a9c1af8cf814030bde56b67ca5c2fbe: ending tablet copy session dbcdc6ac635b44b185c3eba63f66e9d6-0922994ff62d4e648d5132cb08b6a3c8 on tablet 0922994ff62d4e648d5132cb08b6a3c8 with peer dbcdc6ac635b44b185c3eba63f66e9d6
I20240502 20:46:29.563238 31428 leader_election.cc:302] T d670c6c209764e64b0ba7373dfc1f572 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:29.563372 4619 raft_consensus.cc:2781] T d670c6c209764e64b0ba7373dfc1f572 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:29.563416 4619 raft_consensus.cc:482] T d670c6c209764e64b0ba7373dfc1f572 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:29.563450 4619 raft_consensus.cc:3037] T d670c6c209764e64b0ba7373dfc1f572 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:29.563546 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d670c6c209764e64b0ba7373dfc1f572" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:29.563679 29315 raft_consensus.cc:2454] T d670c6c209764e64b0ba7373dfc1f572 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:29.564375 4619 raft_consensus.cc:504] T d670c6c209764e64b0ba7373dfc1f572 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:29.564682 4619 leader_election.cc:288] T d670c6c209764e64b0ba7373dfc1f572 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:29.564997 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d670c6c209764e64b0ba7373dfc1f572" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:29.565028 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d670c6c209764e64b0ba7373dfc1f572" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:29.565112 31755 raft_consensus.cc:3037] T d670c6c209764e64b0ba7373dfc1f572 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:29.565142 29315 raft_consensus.cc:3037] T d670c6c209764e64b0ba7373dfc1f572 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:29.565913 31755 raft_consensus.cc:2454] T d670c6c209764e64b0ba7373dfc1f572 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:29.566107 29315 raft_consensus.cc:2454] T d670c6c209764e64b0ba7373dfc1f572 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:29.566232 31428 leader_election.cc:302] T d670c6c209764e64b0ba7373dfc1f572 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:29.566391 4619 raft_consensus.cc:2781] T d670c6c209764e64b0ba7373dfc1f572 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:29.566484 4619 raft_consensus.cc:686] T d670c6c209764e64b0ba7373dfc1f572 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Becoming Leader. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Running, Role: LEADER
I20240502 20:46:29.566603 4619 consensus_queue.cc:237] T d670c6c209764e64b0ba7373dfc1f572 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:29.568660 29034 catalog_manager.cc:5484] T d670c6c209764e64b0ba7373dfc1f572 P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69). New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:29.622591 5852 ts_tablet_manager.cc:902] T e7642b18bcbb467187fc8c8b924b79df P 2a9c1af8cf814030bde56b67ca5c2fbe: Initiating tablet copy from peer db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:29.623085 5852 tablet_copy_client.cc:308] T e7642b18bcbb467187fc8c8b924b79df P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.68:45897
I20240502 20:46:29.623533 29597 tablet_copy_service.cc:140] P db17f26a73f3447b92aa9a947e3a7057: Received BeginTabletCopySession request for tablet e7642b18bcbb467187fc8c8b924b79df from peer 2a9c1af8cf814030bde56b67ca5c2fbe ({username='slave'} at 127.22.45.69:37699)
I20240502 20:46:29.623629 29597 tablet_copy_service.cc:161] P db17f26a73f3447b92aa9a947e3a7057: Beginning new tablet copy session on tablet e7642b18bcbb467187fc8c8b924b79df from peer 2a9c1af8cf814030bde56b67ca5c2fbe at {username='slave'} at 127.22.45.69:37699: session id = 2a9c1af8cf814030bde56b67ca5c2fbe-e7642b18bcbb467187fc8c8b924b79df
I20240502 20:46:29.624459 29597 tablet_copy_source_session.cc:205] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:29.625221 5852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7642b18bcbb467187fc8c8b924b79df. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:29.627259 5852 tablet_copy_client.cc:783] T e7642b18bcbb467187fc8c8b924b79df P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:29.627477 5852 tablet_copy_client.cc:647] T e7642b18bcbb467187fc8c8b924b79df P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:29.628679 5852 tablet_copy_client.cc:523] T e7642b18bcbb467187fc8c8b924b79df P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:29.629776 5852 tablet_bootstrap.cc:492] T e7642b18bcbb467187fc8c8b924b79df P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:29.644508 5858 ts_tablet_manager.cc:902] T 3943d49acd0341c1a923d900281f1591 P e6b60803d0764d228c863e2da65e3691: Initiating tablet copy from peer 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:29.645018 5858 tablet_copy_client.cc:308] T 3943d49acd0341c1a923d900281f1591 P e6b60803d0764d228c863e2da65e3691: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.65:43267
I20240502 20:46:29.645437 29202 tablet_copy_service.cc:140] P 7796ea3fba294d17a34386873969431d: Received BeginTabletCopySession request for tablet 3943d49acd0341c1a923d900281f1591 from peer e6b60803d0764d228c863e2da65e3691 ({username='slave'} at 127.22.45.72:55003)
I20240502 20:46:29.645541 29202 tablet_copy_service.cc:161] P 7796ea3fba294d17a34386873969431d: Beginning new tablet copy session on tablet 3943d49acd0341c1a923d900281f1591 from peer e6b60803d0764d228c863e2da65e3691 at {username='slave'} at 127.22.45.72:55003: session id = e6b60803d0764d228c863e2da65e3691-3943d49acd0341c1a923d900281f1591
I20240502 20:46:29.646400 29202 tablet_copy_source_session.cc:205] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:29.647265 5858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3943d49acd0341c1a923d900281f1591. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:29.648350 5852 tablet_bootstrap.cc:492] T e7642b18bcbb467187fc8c8b924b79df P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:29.648595 5852 tablet_bootstrap.cc:492] T e7642b18bcbb467187fc8c8b924b79df P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap complete.
I20240502 20:46:29.648751 5852 ts_tablet_manager.cc:1367] T e7642b18bcbb467187fc8c8b924b79df P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.019s user 0.014s sys 0.003s
I20240502 20:46:29.649113 5852 raft_consensus.cc:348] T e7642b18bcbb467187fc8c8b924b79df P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:29.649174 5858 tablet_copy_client.cc:783] T 3943d49acd0341c1a923d900281f1591 P e6b60803d0764d228c863e2da65e3691: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:29.649221 5852 raft_consensus.cc:729] T e7642b18bcbb467187fc8c8b924b79df P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:29.649309 5858 tablet_copy_client.cc:647] T 3943d49acd0341c1a923d900281f1591 P e6b60803d0764d228c863e2da65e3691: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:29.649375 5852 consensus_queue.cc:260] T e7642b18bcbb467187fc8c8b924b79df P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:29.649677 5852 ts_tablet_manager.cc:1398] T e7642b18bcbb467187fc8c8b924b79df P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 20:46:29.650276 29597 tablet_copy_service.cc:342] P db17f26a73f3447b92aa9a947e3a7057: Request end of tablet copy session 2a9c1af8cf814030bde56b67ca5c2fbe-e7642b18bcbb467187fc8c8b924b79df received from {username='slave'} at 127.22.45.69:37699
I20240502 20:46:29.650346 29597 tablet_copy_service.cc:433] P db17f26a73f3447b92aa9a947e3a7057: ending tablet copy session 2a9c1af8cf814030bde56b67ca5c2fbe-e7642b18bcbb467187fc8c8b924b79df on tablet e7642b18bcbb467187fc8c8b924b79df with peer 2a9c1af8cf814030bde56b67ca5c2fbe
I20240502 20:46:29.651336 5858 tablet_copy_client.cc:523] T 3943d49acd0341c1a923d900281f1591 P e6b60803d0764d228c863e2da65e3691: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:29.652576 5858 tablet_bootstrap.cc:492] T 3943d49acd0341c1a923d900281f1591 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:29.662360 29034 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:29.666302 31624 raft_consensus.cc:1261] T 04ebdefddc1349c586b5242a6501661d P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:29.666721 5843 consensus_queue.cc:1035] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:29.666846 29182 raft_consensus.cc:1261] T 04ebdefddc1349c586b5242a6501661d P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:29.667312 5843 consensus_queue.cc:1035] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:29.669503 29034 catalog_manager.cc:5576] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751) does not have the latest schema for tablet 04ebdefddc1349c586b5242a6501661d (table rebalancer_dynamic_table [id=34c07c8d7d144b81946dee5ca2537aa7]). Expected version 1 got 0
I20240502 20:46:29.670655 29041 catalog_manager.cc:5576] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267) does not have the latest schema for tablet 04ebdefddc1349c586b5242a6501661d (table rebalancer_dynamic_table [id=34c07c8d7d144b81946dee5ca2537aa7]). Expected version 1 got 0
I20240502 20:46:29.673979 5870 tablet.cc:1722] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:29.674296 5870 tablet.cc:1620] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13: MemRowSet was empty: no flush needed.
I20240502 20:46:29.675151 5858 tablet_bootstrap.cc:492] T 3943d49acd0341c1a923d900281f1591 P e6b60803d0764d228c863e2da65e3691: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:29.675401 5858 tablet_bootstrap.cc:492] T 3943d49acd0341c1a923d900281f1591 P e6b60803d0764d228c863e2da65e3691: Bootstrap complete.
I20240502 20:46:29.675585 5858 ts_tablet_manager.cc:1367] T 3943d49acd0341c1a923d900281f1591 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.023s user 0.017s sys 0.005s
I20240502 20:46:29.676018 5858 raft_consensus.cc:348] T 3943d49acd0341c1a923d900281f1591 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:29.676127 5858 raft_consensus.cc:729] T 3943d49acd0341c1a923d900281f1591 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:29.676189 29034 catalog_manager.cc:5576] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751) does not have the latest schema for tablet 04ebdefddc1349c586b5242a6501661d (table rebalancer_dynamic_table [id=34c07c8d7d144b81946dee5ca2537aa7]). Expected version 1 got 0
I20240502 20:46:29.676239 5858 consensus_queue.cc:260] T 3943d49acd0341c1a923d900281f1591 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:29.676538 5858 ts_tablet_manager.cc:1398] T 3943d49acd0341c1a923d900281f1591 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:29.677397 5871 tablet.cc:1722] T 04ebdefddc1349c586b5242a6501661d P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:29.677819 5871 tablet.cc:1620] T 04ebdefddc1349c586b5242a6501661d P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:29.678690 29202 tablet_copy_service.cc:342] P 7796ea3fba294d17a34386873969431d: Request end of tablet copy session e6b60803d0764d228c863e2da65e3691-3943d49acd0341c1a923d900281f1591 received from {username='slave'} at 127.22.45.72:55003
I20240502 20:46:29.678769 29202 tablet_copy_service.cc:433] P 7796ea3fba294d17a34386873969431d: ending tablet copy session e6b60803d0764d228c863e2da65e3691-3943d49acd0341c1a923d900281f1591 on tablet 3943d49acd0341c1a923d900281f1591 with peer e6b60803d0764d228c863e2da65e3691
I20240502 20:46:29.682267 5872 tablet.cc:1722] T 04ebdefddc1349c586b5242a6501661d P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:29.682941 5872 tablet.cc:1620] T 04ebdefddc1349c586b5242a6501661d P 7796ea3fba294d17a34386873969431d: MemRowSet was empty: no flush needed.
I20240502 20:46:29.682971 29041 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=34c07c8d7d144b81946dee5ca2537aa7] alter complete (version 1)
I20240502 20:46:29.683085 29042 catalog_manager.cc:5576] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267) does not have the latest schema for tablet 04ebdefddc1349c586b5242a6501661d (table rebalancer_dynamic_table [id=34c07c8d7d144b81946dee5ca2537aa7]). Expected version 1 got 0
I20240502 20:46:29.722074 5656 raft_consensus.cc:482] T 0b9f8b50d5df4c57b5bf058427befec1 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:29.722193 5656 raft_consensus.cc:504] T 0b9f8b50d5df4c57b5bf058427befec1 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:29.722667 5656 leader_election.cc:288] T 0b9f8b50d5df4c57b5bf058427befec1 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:29.722982 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9f8b50d5df4c57b5bf058427befec1" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:29.723124 31495 raft_consensus.cc:2454] T 0b9f8b50d5df4c57b5bf058427befec1 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:29.723093 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9f8b50d5df4c57b5bf058427befec1" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:29.723213 31755 raft_consensus.cc:2454] T 0b9f8b50d5df4c57b5bf058427befec1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:29.723428 29119 leader_election.cc:302] T 0b9f8b50d5df4c57b5bf058427befec1 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:29.723604 5656 raft_consensus.cc:2781] T 0b9f8b50d5df4c57b5bf058427befec1 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:29.723660 5656 raft_consensus.cc:482] T 0b9f8b50d5df4c57b5bf058427befec1 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:29.723695 5656 raft_consensus.cc:3037] T 0b9f8b50d5df4c57b5bf058427befec1 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:29.724498 5656 raft_consensus.cc:504] T 0b9f8b50d5df4c57b5bf058427befec1 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:29.724802 5656 leader_election.cc:288] T 0b9f8b50d5df4c57b5bf058427befec1 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:29.725086 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9f8b50d5df4c57b5bf058427befec1" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:29.725203 31755 raft_consensus.cc:3037] T 0b9f8b50d5df4c57b5bf058427befec1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:29.725260 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9f8b50d5df4c57b5bf058427befec1" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:29.725380 31495 raft_consensus.cc:3037] T 0b9f8b50d5df4c57b5bf058427befec1 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:29.726099 31755 raft_consensus.cc:2454] T 0b9f8b50d5df4c57b5bf058427befec1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:29.726361 29117 leader_election.cc:302] T 0b9f8b50d5df4c57b5bf058427befec1 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:29.726372 31495 raft_consensus.cc:2454] T 0b9f8b50d5df4c57b5bf058427befec1 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:29.726605 5656 raft_consensus.cc:2781] T 0b9f8b50d5df4c57b5bf058427befec1 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:29.726692 5656 raft_consensus.cc:686] T 0b9f8b50d5df4c57b5bf058427befec1 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:29.726886 5656 consensus_queue.cc:237] T 0b9f8b50d5df4c57b5bf058427befec1 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:29.729209 29034 catalog_manager.cc:5484] T 0b9f8b50d5df4c57b5bf058427befec1 P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:29.741312 5471 raft_consensus.cc:482] T 653cbec396434b4dac0a1928b21d0ebd P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:29.741407 5471 raft_consensus.cc:504] T 653cbec396434b4dac0a1928b21d0ebd P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:29.741744 5471 leader_election.cc:288] T 653cbec396434b4dac0a1928b21d0ebd P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:29.742136 31887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "653cbec396434b4dac0a1928b21d0ebd" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:29.742229 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "653cbec396434b4dac0a1928b21d0ebd" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:29.742272 31887 raft_consensus.cc:2454] T 653cbec396434b4dac0a1928b21d0ebd P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:29.742357 31625 raft_consensus.cc:2454] T 653cbec396434b4dac0a1928b21d0ebd P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:29.742682 29380 leader_election.cc:302] T 653cbec396434b4dac0a1928b21d0ebd P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:29.743482 5471 raft_consensus.cc:2781] T 653cbec396434b4dac0a1928b21d0ebd P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:29.743542 5471 raft_consensus.cc:482] T 653cbec396434b4dac0a1928b21d0ebd P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:29.743579 5471 raft_consensus.cc:3037] T 653cbec396434b4dac0a1928b21d0ebd P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:29.744549 5471 raft_consensus.cc:504] T 653cbec396434b4dac0a1928b21d0ebd P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:29.744879 5471 leader_election.cc:288] T 653cbec396434b4dac0a1928b21d0ebd P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:29.745211 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "653cbec396434b4dac0a1928b21d0ebd" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:29.745317 31625 raft_consensus.cc:3037] T 653cbec396434b4dac0a1928b21d0ebd P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:29.745283 31886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "653cbec396434b4dac0a1928b21d0ebd" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:29.745393 31886 raft_consensus.cc:3037] T 653cbec396434b4dac0a1928b21d0ebd P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:29.746297 31625 raft_consensus.cc:2454] T 653cbec396434b4dac0a1928b21d0ebd P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:29.746395 31886 raft_consensus.cc:2454] T 653cbec396434b4dac0a1928b21d0ebd P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:29.746555 29380 leader_election.cc:302] T 653cbec396434b4dac0a1928b21d0ebd P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:29.746873 5471 raft_consensus.cc:2781] T 653cbec396434b4dac0a1928b21d0ebd P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:29.746966 5471 raft_consensus.cc:686] T 653cbec396434b4dac0a1928b21d0ebd P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:29.747092 5471 consensus_queue.cc:237] T 653cbec396434b4dac0a1928b21d0ebd P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:29.749083 29034 catalog_manager.cc:5484] T 653cbec396434b4dac0a1928b21d0ebd P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:29.775199 31605 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:29.776221 29162 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 20:46:29.782589 29295 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:29.785777 29034 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:29.785946 31868 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:29.798831 31735 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:29.798990 5870 tablet.cc:1722] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:29.799328 5870 tablet.cc:1620] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13: MemRowSet was empty: no flush needed.
I20240502 20:46:29.800141 5871 tablet.cc:1722] T 04ebdefddc1349c586b5242a6501661d P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:29.800436 5871 tablet.cc:1620] T 04ebdefddc1349c586b5242a6501661d P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:29.801905 29425 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:29.808104 31474 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:29.812440 5872 tablet.cc:1722] T 04ebdefddc1349c586b5242a6501661d P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:29.812749 5872 tablet.cc:1620] T 04ebdefddc1349c586b5242a6501661d P 7796ea3fba294d17a34386873969431d: MemRowSet was empty: no flush needed.
I20240502 20:46:29.819257 29042 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=34c07c8d7d144b81946dee5ca2537aa7] alter complete (version 2)
I20240502 20:46:29.855758 29556 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:29.902878 29182 consensus_queue.cc:237] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:29.904294 31624 raft_consensus.cc:1261] T b28fe6bccf4f48a6a5456e6ead5cc920 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:29.904726 29574 raft_consensus.cc:1261] T b28fe6bccf4f48a6a5456e6ead5cc920 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:29.905447 5656 consensus_queue.cc:1035] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:29.905771 5656 consensus_queue.cc:1035] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:29.914502 5656 raft_consensus.cc:2932] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:29.915766 29574 raft_consensus.cc:2932] T b28fe6bccf4f48a6a5456e6ead5cc920 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:29.918275 29034 catalog_manager.cc:5484] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d reported cstate change: config changed from index 4 to 5, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) evicted. New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:29.920619 31624 raft_consensus.cc:2932] T b28fe6bccf4f48a6a5456e6ead5cc920 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:29.921597 29295 tablet_service.cc:1514] Processing DeleteTablet for tablet b28fe6bccf4f48a6a5456e6ead5cc920 with delete_type TABLET_DATA_TOMBSTONED (TS 09258ac095664fa99caf0707262cde13 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:29.921921 5913 tablet_replica.cc:330] T b28fe6bccf4f48a6a5456e6ead5cc920 P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:29.922902 5913 raft_consensus.cc:2229] T b28fe6bccf4f48a6a5456e6ead5cc920 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:29.923112 5913 raft_consensus.cc:2258] T b28fe6bccf4f48a6a5456e6ead5cc920 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:29.923664 29041 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:29.927671 5913 ts_tablet_manager.cc:1875] T b28fe6bccf4f48a6a5456e6ead5cc920 P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:29.929993 5870 tablet.cc:1722] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:29.930383 5871 tablet.cc:1722] T 04ebdefddc1349c586b5242a6501661d P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:29.931746 5872 tablet.cc:1722] T 04ebdefddc1349c586b5242a6501661d P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:29.939152 29041 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=34c07c8d7d144b81946dee5ca2537aa7] alter complete (version 3)
I20240502 20:46:29.942528 5913 ts_tablet_manager.cc:1888] T b28fe6bccf4f48a6a5456e6ead5cc920 P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 20:46:29.942651 5913 log.cc:1196] T b28fe6bccf4f48a6a5456e6ead5cc920 P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/b28fe6bccf4f48a6a5456e6ead5cc920
I20240502 20:46:29.943132 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet b28fe6bccf4f48a6a5456e6ead5cc920 (table rebalance_test_table_19 [id=5fd0afffa41641b9b8de6545456ff39a]) successfully deleted
I20240502 20:46:30.042225 29035 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:30.042372 29035 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:30.043927 29162 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 20:46:30.045439 29035 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 04ebdefddc1349c586b5242a6501661d
I20240502 20:46:30.046226 31605 tablet_service.cc:1514] Processing DeleteTablet for tablet 04ebdefddc1349c586b5242a6501661d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:30 UTC) from {username='slave'} at 127.0.0.1:58346
I20240502 20:46:30.046289 29295 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:30.046450 29294 tablet_service.cc:1514] Processing DeleteTablet for tablet 04ebdefddc1349c586b5242a6501661d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:30 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:30.046640 5913 tablet_replica.cc:330] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:30.046634 29162 tablet_service.cc:1514] Processing DeleteTablet for tablet 04ebdefddc1349c586b5242a6501661d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:30 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:30.046772 5913 raft_consensus.cc:2229] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:30.046921 31474 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:30.046978 5913 raft_consensus.cc:2258] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:30.047775 5913 ts_tablet_manager.cc:1875] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:30.049903 5913 ts_tablet_manager.cc:1888] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:30.049985 5913 log.cc:1196] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/04ebdefddc1349c586b5242a6501661d
I20240502 20:46:30.050231 5913 ts_tablet_manager.cc:1909] T 04ebdefddc1349c586b5242a6501661d P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:30.052865 5962 tablet_replica.cc:330] T 04ebdefddc1349c586b5242a6501661d P 315915efcf2b413ca7f9d874338e1b2d: stopping tablet replica
I20240502 20:46:30.053001 31605 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:30.053099 5962 raft_consensus.cc:2229] T 04ebdefddc1349c586b5242a6501661d P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:30.053254 5962 raft_consensus.cc:2258] T 04ebdefddc1349c586b5242a6501661d P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:30.053747 5963 tablet_replica.cc:330] T 04ebdefddc1349c586b5242a6501661d P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:30.053927 5963 raft_consensus.cc:2229] T 04ebdefddc1349c586b5242a6501661d P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:30.054065 5963 raft_consensus.cc:2258] T 04ebdefddc1349c586b5242a6501661d P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:30.054924 5963 ts_tablet_manager.cc:1875] T 04ebdefddc1349c586b5242a6501661d P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:30.054939 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet 04ebdefddc1349c586b5242a6501661d (table rebalancer_dynamic_table_new_name [id=34c07c8d7d144b81946dee5ca2537aa7]) successfully deleted
I20240502 20:46:30.059237 5963 ts_tablet_manager.cc:1888] T 04ebdefddc1349c586b5242a6501661d P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:30.059350 5963 log.cc:1196] T 04ebdefddc1349c586b5242a6501661d P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/04ebdefddc1349c586b5242a6501661d
I20240502 20:46:30.059867 5963 ts_tablet_manager.cc:1909] T 04ebdefddc1349c586b5242a6501661d P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:30.063093 5962 ts_tablet_manager.cc:1875] T 04ebdefddc1349c586b5242a6501661d P 315915efcf2b413ca7f9d874338e1b2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:30.064064 29556 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:30.066048 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 04ebdefddc1349c586b5242a6501661d (table rebalancer_dynamic_table_new_name [id=34c07c8d7d144b81946dee5ca2537aa7]) successfully deleted
I20240502 20:46:30.066301 5962 ts_tablet_manager.cc:1888] T 04ebdefddc1349c586b5242a6501661d P 315915efcf2b413ca7f9d874338e1b2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:30.066396 5962 log.cc:1196] T 04ebdefddc1349c586b5242a6501661d P 315915efcf2b413ca7f9d874338e1b2d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal/wals/04ebdefddc1349c586b5242a6501661d
I20240502 20:46:30.066962 5962 ts_tablet_manager.cc:1909] T 04ebdefddc1349c586b5242a6501661d P 315915efcf2b413ca7f9d874338e1b2d: Deleting consensus metadata
I20240502 20:46:30.068153 29027 catalog_manager.cc:4830] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): tablet 04ebdefddc1349c586b5242a6501661d (table rebalancer_dynamic_table_new_name [id=34c07c8d7d144b81946dee5ca2537aa7]) successfully deleted
I20240502 20:46:30.075321 31868 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:30.098752 29425 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:30.101119 31735 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:30.106474 5655 consensus_queue.cc:1035] T 0b9f8b50d5df4c57b5bf058427befec1 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:30.111752 5656 consensus_queue.cc:1035] T 0b9f8b50d5df4c57b5bf058427befec1 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:30.119390 29446 raft_consensus.cc:1203] T 0922994ff62d4e648d5132cb08b6a3c8 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 20:46:30.140388 4619 consensus_queue.cc:1035] T d670c6c209764e64b0ba7373dfc1f572 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:30.142403 29182 consensus_queue.cc:237] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:30.143260 4619 consensus_queue.cc:1035] T d670c6c209764e64b0ba7373dfc1f572 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:30.143944 31495 raft_consensus.cc:1261] T 849c31557dec4043ad36dbddc6811b3f P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:30.144376 5655 consensus_queue.cc:1035] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:30.146194 29574 raft_consensus.cc:1261] T 849c31557dec4043ad36dbddc6811b3f P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:30.146250 5656 raft_consensus.cc:2932] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:30.146572 5655 consensus_queue.cc:1035] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:30.147696 29574 raft_consensus.cc:2932] T 849c31557dec4043ad36dbddc6811b3f P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:30.149391 29038 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:30.149632 29036 catalog_manager.cc:5484] T 849c31557dec4043ad36dbddc6811b3f P 7796ea3fba294d17a34386873969431d reported cstate change: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:30.151240 31495 raft_consensus.cc:2932] T 849c31557dec4043ad36dbddc6811b3f P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:30.153934 29425 tablet_service.cc:1514] Processing DeleteTablet for tablet 849c31557dec4043ad36dbddc6811b3f with delete_type TABLET_DATA_TOMBSTONED (TS dbcdc6ac635b44b185c3eba63f66e9d6 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:30.155918 29162 tablet_service.cc:1467] Processing CreateTablet for tablet 3d6220f24edd4fdc826a6cccabf599eb (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a879a41eab5f441da53f51e971c97f42]), partition=
I20240502 20:46:30.156147 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d6220f24edd4fdc826a6cccabf599eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:30.156349 5990 tablet_replica.cc:330] T 849c31557dec4043ad36dbddc6811b3f P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:30.156488 5990 raft_consensus.cc:2229] T 849c31557dec4043ad36dbddc6811b3f P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:30.156565 31605 tablet_service.cc:1467] Processing CreateTablet for tablet 3d6220f24edd4fdc826a6cccabf599eb (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a879a41eab5f441da53f51e971c97f42]), partition=
I20240502 20:46:30.156644 5990 raft_consensus.cc:2258] T 849c31557dec4043ad36dbddc6811b3f P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:30.156772 31605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d6220f24edd4fdc826a6cccabf599eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:30.157399 5990 ts_tablet_manager.cc:1875] T 849c31557dec4043ad36dbddc6811b3f P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:30.159258 29294 tablet_service.cc:1467] Processing CreateTablet for tablet 3d6220f24edd4fdc826a6cccabf599eb (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a879a41eab5f441da53f51e971c97f42]), partition=
I20240502 20:46:30.159456 5996 tablet_bootstrap.cc:492] T 3d6220f24edd4fdc826a6cccabf599eb P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:30.159489 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d6220f24edd4fdc826a6cccabf599eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:30.160501 5996 tablet_bootstrap.cc:654] T 3d6220f24edd4fdc826a6cccabf599eb P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:30.160759 5990 ts_tablet_manager.cc:1888] T 849c31557dec4043ad36dbddc6811b3f P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 20:46:30.160811 5990 log.cc:1196] T 849c31557dec4043ad36dbddc6811b3f P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/849c31557dec4043ad36dbddc6811b3f
I20240502 20:46:30.161216 5992 tablet_bootstrap.cc:492] T 3d6220f24edd4fdc826a6cccabf599eb P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:30.162009 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet 849c31557dec4043ad36dbddc6811b3f (table rebalance_test_table_17 [id=b1432bc7242c45e38bf75a9c91c6646c]) successfully deleted
I20240502 20:46:30.162237 5996 tablet_bootstrap.cc:492] T 3d6220f24edd4fdc826a6cccabf599eb P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:30.162329 5996 ts_tablet_manager.cc:1367] T 3d6220f24edd4fdc826a6cccabf599eb P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:30.162516 5998 tablet_bootstrap.cc:492] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:30.162786 5996 raft_consensus.cc:348] T 3d6220f24edd4fdc826a6cccabf599eb P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:30.162918 5996 raft_consensus.cc:374] T 3d6220f24edd4fdc826a6cccabf599eb P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:30.163005 5996 raft_consensus.cc:729] T 3d6220f24edd4fdc826a6cccabf599eb P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:30.163131 5996 consensus_queue.cc:260] T 3d6220f24edd4fdc826a6cccabf599eb P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:30.163411 5998 tablet_bootstrap.cc:654] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:30.163516 5996 ts_tablet_manager.cc:1398] T 3d6220f24edd4fdc826a6cccabf599eb P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:30.163854 5992 tablet_bootstrap.cc:654] T 3d6220f24edd4fdc826a6cccabf599eb P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:30.165066 5992 tablet_bootstrap.cc:492] T 3d6220f24edd4fdc826a6cccabf599eb P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:30.165153 5992 ts_tablet_manager.cc:1367] T 3d6220f24edd4fdc826a6cccabf599eb P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:30.165601 5992 raft_consensus.cc:348] T 3d6220f24edd4fdc826a6cccabf599eb P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:30.165676 5992 raft_consensus.cc:374] T 3d6220f24edd4fdc826a6cccabf599eb P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:30.165704 5992 raft_consensus.cc:729] T 3d6220f24edd4fdc826a6cccabf599eb P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:30.165818 5992 consensus_queue.cc:260] T 3d6220f24edd4fdc826a6cccabf599eb P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:30.166088 5992 ts_tablet_manager.cc:1398] T 3d6220f24edd4fdc826a6cccabf599eb P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:30.168951 5998 tablet_bootstrap.cc:492] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:30.169070 5998 ts_tablet_manager.cc:1367] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 20:46:30.169461 5998 raft_consensus.cc:348] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:30.169550 5998 raft_consensus.cc:374] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:30.169579 5998 raft_consensus.cc:729] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:30.169698 5998 consensus_queue.cc:260] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:30.169939 5998 ts_tablet_manager.cc:1398] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:30.174772 31886 raft_consensus.cc:1203] T 3943d49acd0341c1a923d900281f1591 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:30.175196 31495 raft_consensus.cc:1203] T e7642b18bcbb467187fc8c8b924b79df P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:30.179852 4694 consensus_queue.cc:1035] T 653cbec396434b4dac0a1928b21d0ebd P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:30.182654 4694 consensus_queue.cc:1035] T 653cbec396434b4dac0a1928b21d0ebd P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 20:46:30.233999 29231 tablet.cc:2369] T 3d6220f24edd4fdc826a6cccabf599eb P 7796ea3fba294d17a34386873969431d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 20:46:30.312716 29039 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_46"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:30.318480 29556 tablet_service.cc:1467] Processing CreateTablet for tablet f0188e48270242949913034b04dbd3b0 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=2fff9f5acb864bcab32ed0faaef818bb]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:30.318776 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0188e48270242949913034b04dbd3b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:30.319195 31474 tablet_service.cc:1467] Processing CreateTablet for tablet aca63a2154fc4f688fd3a254b572de54 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=2fff9f5acb864bcab32ed0faaef818bb]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:30.319240 29425 tablet_service.cc:1467] Processing CreateTablet for tablet 9da27cc9c2504b7899da253a88dd1f56 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=2fff9f5acb864bcab32ed0faaef818bb]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:30.319469 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9da27cc9c2504b7899da253a88dd1f56. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:30.319485 31474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aca63a2154fc4f688fd3a254b572de54. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:30.321813 29294 tablet_service.cc:1467] Processing CreateTablet for tablet f0188e48270242949913034b04dbd3b0 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=2fff9f5acb864bcab32ed0faaef818bb]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:30.322053 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0188e48270242949913034b04dbd3b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:30.322026 31605 tablet_service.cc:1467] Processing CreateTablet for tablet f0188e48270242949913034b04dbd3b0 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=2fff9f5acb864bcab32ed0faaef818bb]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:30.322256 31605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0188e48270242949913034b04dbd3b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:30.322450 31602 tablet_service.cc:1467] Processing CreateTablet for tablet 9da27cc9c2504b7899da253a88dd1f56 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=2fff9f5acb864bcab32ed0faaef818bb]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:30.322441 29556 tablet_service.cc:1467] Processing CreateTablet for tablet 9da27cc9c2504b7899da253a88dd1f56 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=2fff9f5acb864bcab32ed0faaef818bb]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:30.322681 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9da27cc9c2504b7899da253a88dd1f56. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:30.322690 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9da27cc9c2504b7899da253a88dd1f56. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:30.324169 5998 tablet_bootstrap.cc:492] T f0188e48270242949913034b04dbd3b0 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:30.321748 6042 tablet_bootstrap.cc:492] T f0188e48270242949913034b04dbd3b0 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:30.324606 5996 tablet_bootstrap.cc:492] T f0188e48270242949913034b04dbd3b0 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:30.325095 5998 tablet_bootstrap.cc:654] T f0188e48270242949913034b04dbd3b0 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:30.325099 6042 tablet_bootstrap.cc:654] T f0188e48270242949913034b04dbd3b0 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:30.325534 5996 tablet_bootstrap.cc:654] T f0188e48270242949913034b04dbd3b0 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:30.325547 6043 tablet_bootstrap.cc:492] T aca63a2154fc4f688fd3a254b572de54 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:30.326480 31868 tablet_service.cc:1467] Processing CreateTablet for tablet aca63a2154fc4f688fd3a254b572de54 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=2fff9f5acb864bcab32ed0faaef818bb]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:30.326711 6043 tablet_bootstrap.cc:654] T aca63a2154fc4f688fd3a254b572de54 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:30.326745 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aca63a2154fc4f688fd3a254b572de54. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:30.326751 5996 tablet_bootstrap.cc:492] T f0188e48270242949913034b04dbd3b0 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:30.326819 5996 ts_tablet_manager.cc:1367] T f0188e48270242949913034b04dbd3b0 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 20:46:30.327008 6044 tablet_bootstrap.cc:492] T 9da27cc9c2504b7899da253a88dd1f56 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:30.327261 5996 raft_consensus.cc:348] T f0188e48270242949913034b04dbd3b0 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:30.327358 5996 raft_consensus.cc:374] T f0188e48270242949913034b04dbd3b0 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:30.327387 5996 raft_consensus.cc:729] T f0188e48270242949913034b04dbd3b0 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:30.327504 5996 consensus_queue.cc:260] T f0188e48270242949913034b04dbd3b0 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:30.327775 5996 ts_tablet_manager.cc:1398] T f0188e48270242949913034b04dbd3b0 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:30.327885 6044 tablet_bootstrap.cc:654] T 9da27cc9c2504b7899da253a88dd1f56 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:30.327936 5996 tablet_bootstrap.cc:492] T 9da27cc9c2504b7899da253a88dd1f56 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:30.328728 5996 tablet_bootstrap.cc:654] T 9da27cc9c2504b7899da253a88dd1f56 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:30.329025 29425 tablet_service.cc:1467] Processing CreateTablet for tablet aca63a2154fc4f688fd3a254b572de54 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=2fff9f5acb864bcab32ed0faaef818bb]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:30.329248 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aca63a2154fc4f688fd3a254b572de54. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:30.329264 6043 tablet_bootstrap.cc:492] T aca63a2154fc4f688fd3a254b572de54 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:30.329339 6043 ts_tablet_manager.cc:1367] T aca63a2154fc4f688fd3a254b572de54 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:30.329517 6048 tablet_bootstrap.cc:492] T aca63a2154fc4f688fd3a254b572de54 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:30.329733 6043 raft_consensus.cc:348] T aca63a2154fc4f688fd3a254b572de54 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:30.329830 6043 raft_consensus.cc:374] T aca63a2154fc4f688fd3a254b572de54 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:30.329857 6043 raft_consensus.cc:729] T aca63a2154fc4f688fd3a254b572de54 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:30.329993 6043 consensus_queue.cc:260] T aca63a2154fc4f688fd3a254b572de54 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:30.330277 6043 ts_tablet_manager.cc:1398] T aca63a2154fc4f688fd3a254b572de54 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:30.330367 6048 tablet_bootstrap.cc:654] T aca63a2154fc4f688fd3a254b572de54 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:30.330978 5998 tablet_bootstrap.cc:492] T f0188e48270242949913034b04dbd3b0 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:30.331063 5998 ts_tablet_manager.cc:1367] T f0188e48270242949913034b04dbd3b0 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 20:46:30.331475 5998 raft_consensus.cc:348] T f0188e48270242949913034b04dbd3b0 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:30.331545 6044 tablet_bootstrap.cc:492] T 9da27cc9c2504b7899da253a88dd1f56 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:30.331570 5998 raft_consensus.cc:374] T f0188e48270242949913034b04dbd3b0 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:30.331598 5998 raft_consensus.cc:729] T f0188e48270242949913034b04dbd3b0 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:30.331624 6044 ts_tablet_manager.cc:1367] T 9da27cc9c2504b7899da253a88dd1f56 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 20:46:30.331722 5998 consensus_queue.cc:260] T f0188e48270242949913034b04dbd3b0 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:30.332019 5998 ts_tablet_manager.cc:1398] T f0188e48270242949913034b04dbd3b0 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:30.332026 6044 raft_consensus.cc:348] T 9da27cc9c2504b7899da253a88dd1f56 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:30.332104 6044 raft_consensus.cc:374] T 9da27cc9c2504b7899da253a88dd1f56 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:30.332130 6044 raft_consensus.cc:729] T 9da27cc9c2504b7899da253a88dd1f56 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:30.332249 6044 consensus_queue.cc:260] T 9da27cc9c2504b7899da253a88dd1f56 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:30.332525 6044 ts_tablet_manager.cc:1398] T 9da27cc9c2504b7899da253a88dd1f56 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:30.332700 6044 tablet_bootstrap.cc:492] T aca63a2154fc4f688fd3a254b572de54 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:30.333530 6044 tablet_bootstrap.cc:654] T aca63a2154fc4f688fd3a254b572de54 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:30.337250 6042 tablet_bootstrap.cc:492] T f0188e48270242949913034b04dbd3b0 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:30.337395 6042 ts_tablet_manager.cc:1367] T f0188e48270242949913034b04dbd3b0 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240502 20:46:30.337844 6042 raft_consensus.cc:348] T f0188e48270242949913034b04dbd3b0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:30.337935 6042 raft_consensus.cc:374] T f0188e48270242949913034b04dbd3b0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:30.337976 6042 raft_consensus.cc:729] T f0188e48270242949913034b04dbd3b0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:30.338080 6042 consensus_queue.cc:260] T f0188e48270242949913034b04dbd3b0 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:30.338354 6042 ts_tablet_manager.cc:1398] T f0188e48270242949913034b04dbd3b0 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:30.338528 6042 tablet_bootstrap.cc:492] T 9da27cc9c2504b7899da253a88dd1f56 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:30.339656 6042 tablet_bootstrap.cc:654] T 9da27cc9c2504b7899da253a88dd1f56 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
W20240502 20:46:30.340202 29493 tablet.cc:2369] T 9da27cc9c2504b7899da253a88dd1f56 P dbcdc6ac635b44b185c3eba63f66e9d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 20:46:30.340848 6048 tablet_bootstrap.cc:492] T aca63a2154fc4f688fd3a254b572de54 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:30.340898 6042 tablet_bootstrap.cc:492] T 9da27cc9c2504b7899da253a88dd1f56 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:30.340936 6048 ts_tablet_manager.cc:1367] T aca63a2154fc4f688fd3a254b572de54 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 20:46:30.340967 6042 ts_tablet_manager.cc:1367] T 9da27cc9c2504b7899da253a88dd1f56 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:30.341357 6042 raft_consensus.cc:348] T 9da27cc9c2504b7899da253a88dd1f56 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:30.341388 6048 raft_consensus.cc:348] T aca63a2154fc4f688fd3a254b572de54 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:30.341447 6042 raft_consensus.cc:374] T 9da27cc9c2504b7899da253a88dd1f56 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:30.341475 6042 raft_consensus.cc:729] T 9da27cc9c2504b7899da253a88dd1f56 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:30.341480 6048 raft_consensus.cc:374] T aca63a2154fc4f688fd3a254b572de54 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:30.341511 6048 raft_consensus.cc:729] T aca63a2154fc4f688fd3a254b572de54 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:30.341609 6042 consensus_queue.cc:260] T 9da27cc9c2504b7899da253a88dd1f56 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:30.341624 6048 consensus_queue.cc:260] T aca63a2154fc4f688fd3a254b572de54 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:30.341874 6042 ts_tablet_manager.cc:1398] T 9da27cc9c2504b7899da253a88dd1f56 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:30.341889 6048 ts_tablet_manager.cc:1398] T aca63a2154fc4f688fd3a254b572de54 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:30.344709 6044 tablet_bootstrap.cc:492] T aca63a2154fc4f688fd3a254b572de54 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:30.344789 6044 ts_tablet_manager.cc:1367] T aca63a2154fc4f688fd3a254b572de54 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20240502 20:46:30.345326 6044 raft_consensus.cc:348] T aca63a2154fc4f688fd3a254b572de54 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:30.345438 6044 raft_consensus.cc:374] T aca63a2154fc4f688fd3a254b572de54 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:30.345525 6044 raft_consensus.cc:729] T aca63a2154fc4f688fd3a254b572de54 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:30.345655 6044 consensus_queue.cc:260] T aca63a2154fc4f688fd3a254b572de54 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:30.345927 6044 ts_tablet_manager.cc:1398] T aca63a2154fc4f688fd3a254b572de54 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:30.351217 5843 raft_consensus.cc:482] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:30.351308 5843 raft_consensus.cc:504] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:30.351612 5843 leader_election.cc:288] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:30.351941 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d6220f24edd4fdc826a6cccabf599eb" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:30.352069 29182 raft_consensus.cc:2454] T 3d6220f24edd4fdc826a6cccabf599eb P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:30.352272 31624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d6220f24edd4fdc826a6cccabf599eb" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:30.352355 29251 leader_election.cc:302] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:30.352375 31624 raft_consensus.cc:2454] T 3d6220f24edd4fdc826a6cccabf599eb P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:30.352519 5843 raft_consensus.cc:2781] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:30.352576 5843 raft_consensus.cc:482] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:30.352627 5843 raft_consensus.cc:3037] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:30.353585 5843 raft_consensus.cc:504] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:30.353914 5843 leader_election.cc:288] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:30.354202 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d6220f24edd4fdc826a6cccabf599eb" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:30.354293 29182 raft_consensus.cc:3037] T 3d6220f24edd4fdc826a6cccabf599eb P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:30.354840 5996 tablet_bootstrap.cc:492] T 9da27cc9c2504b7899da253a88dd1f56 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:30.354926 5996 ts_tablet_manager.cc:1367] T 9da27cc9c2504b7899da253a88dd1f56 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.027s user 0.000s sys 0.002s
I20240502 20:46:30.355361 29182 raft_consensus.cc:2454] T 3d6220f24edd4fdc826a6cccabf599eb P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:30.355348 5996 raft_consensus.cc:348] T 9da27cc9c2504b7899da253a88dd1f56 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:30.355441 5996 raft_consensus.cc:374] T 9da27cc9c2504b7899da253a88dd1f56 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:30.355490 5996 raft_consensus.cc:729] T 9da27cc9c2504b7899da253a88dd1f56 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:30.355614 5996 consensus_queue.cc:260] T 9da27cc9c2504b7899da253a88dd1f56 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:30.355727 29251 leader_election.cc:302] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:30.355870 5843 raft_consensus.cc:2781] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:30.355958 5843 raft_consensus.cc:686] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:30.355955 5996 ts_tablet_manager.cc:1398] T 9da27cc9c2504b7899da253a88dd1f56 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:30.356026 31624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d6220f24edd4fdc826a6cccabf599eb" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:30.356107 31624 raft_consensus.cc:3037] T 3d6220f24edd4fdc826a6cccabf599eb P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:30.356076 5843 consensus_queue.cc:237] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:30.356814 31624 raft_consensus.cc:2454] T 3d6220f24edd4fdc826a6cccabf599eb P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:30.358742 4694 raft_consensus.cc:482] T aca63a2154fc4f688fd3a254b572de54 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:30.358820 4694 raft_consensus.cc:504] T aca63a2154fc4f688fd3a254b572de54 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:30.359123 4694 leader_election.cc:288] T aca63a2154fc4f688fd3a254b572de54 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:30.359287 29034 catalog_manager.cc:5484] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:30.359458 31887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aca63a2154fc4f688fd3a254b572de54" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:30.359560 31887 raft_consensus.cc:2454] T aca63a2154fc4f688fd3a254b572de54 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:30.359663 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aca63a2154fc4f688fd3a254b572de54" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:30.359772 31495 raft_consensus.cc:2454] T aca63a2154fc4f688fd3a254b572de54 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:30.360054 29380 leader_election.cc:302] T aca63a2154fc4f688fd3a254b572de54 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dbcdc6ac635b44b185c3eba63f66e9d6, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:30.360208 4694 raft_consensus.cc:2781] T aca63a2154fc4f688fd3a254b572de54 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:30.360265 4694 raft_consensus.cc:482] T aca63a2154fc4f688fd3a254b572de54 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:30.360299 4694 raft_consensus.cc:3037] T aca63a2154fc4f688fd3a254b572de54 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:30.361074 4694 raft_consensus.cc:504] T aca63a2154fc4f688fd3a254b572de54 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:30.361415 4694 leader_election.cc:288] T aca63a2154fc4f688fd3a254b572de54 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:30.361666 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aca63a2154fc4f688fd3a254b572de54" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:30.361690 31887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aca63a2154fc4f688fd3a254b572de54" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:30.361771 31495 raft_consensus.cc:3037] T aca63a2154fc4f688fd3a254b572de54 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:30.361778 31887 raft_consensus.cc:3037] T aca63a2154fc4f688fd3a254b572de54 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:30.362675 31495 raft_consensus.cc:2454] T aca63a2154fc4f688fd3a254b572de54 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:30.362674 31887 raft_consensus.cc:2454] T aca63a2154fc4f688fd3a254b572de54 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:30.362918 29381 leader_election.cc:302] T aca63a2154fc4f688fd3a254b572de54 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:30.363123 4694 raft_consensus.cc:2781] T aca63a2154fc4f688fd3a254b572de54 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:30.363215 4694 raft_consensus.cc:686] T aca63a2154fc4f688fd3a254b572de54 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:30.363325 4694 consensus_queue.cc:237] T aca63a2154fc4f688fd3a254b572de54 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:30.365693 29038 catalog_manager.cc:5484] T aca63a2154fc4f688fd3a254b572de54 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:30.378846 29294 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:30.379712 6069 raft_consensus.cc:482] T 9da27cc9c2504b7899da253a88dd1f56 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:30.379791 6069 raft_consensus.cc:504] T 9da27cc9c2504b7899da253a88dd1f56 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:30.381839 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9da27cc9c2504b7899da253a88dd1f56" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:30.381949 29446 raft_consensus.cc:2454] T 9da27cc9c2504b7899da253a88dd1f56 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:30.382270 29512 leader_election.cc:302] T 9da27cc9c2504b7899da253a88dd1f56 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:30.383364 6054 raft_consensus.cc:2781] T 9da27cc9c2504b7899da253a88dd1f56 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:30.383433 6054 raft_consensus.cc:482] T 9da27cc9c2504b7899da253a88dd1f56 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:30.383471 6054 raft_consensus.cc:3037] T 9da27cc9c2504b7899da253a88dd1f56 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:30.384485 6054 raft_consensus.cc:504] T 9da27cc9c2504b7899da253a88dd1f56 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:30.384919 6054 leader_election.cc:288] T 9da27cc9c2504b7899da253a88dd1f56 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Requested vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:30.385097 31624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9da27cc9c2504b7899da253a88dd1f56" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:30.385200 31624 raft_consensus.cc:3037] T 9da27cc9c2504b7899da253a88dd1f56 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:30.385609 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9da27cc9c2504b7899da253a88dd1f56" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:30.385700 29446 raft_consensus.cc:3037] T 9da27cc9c2504b7899da253a88dd1f56 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:30.386188 31624 raft_consensus.cc:2454] T 9da27cc9c2504b7899da253a88dd1f56 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:30.386664 29446 raft_consensus.cc:2454] T 9da27cc9c2504b7899da253a88dd1f56 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:30.386971 29512 leader_election.cc:302] T 9da27cc9c2504b7899da253a88dd1f56 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:30.387874 6054 raft_consensus.cc:2781] T 9da27cc9c2504b7899da253a88dd1f56 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:30.387962 6054 raft_consensus.cc:686] T 9da27cc9c2504b7899da253a88dd1f56 P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Becoming Leader. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Running, Role: LEADER
I20240502 20:46:30.388084 6054 consensus_queue.cc:237] T 9da27cc9c2504b7899da253a88dd1f56 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:30.388422 29556 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
W20240502 20:46:30.388808 29362 tablet.cc:2369] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 20:46:30.391222 4444 raft_consensus.cc:482] T f0188e48270242949913034b04dbd3b0 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:30.391302 4444 raft_consensus.cc:504] T f0188e48270242949913034b04dbd3b0 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:30.391681 4444 leader_election.cc:288] T f0188e48270242949913034b04dbd3b0 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:30.394009 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0188e48270242949913034b04dbd3b0" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:30.394116 29577 raft_consensus.cc:2454] T f0188e48270242949913034b04dbd3b0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:30.394367 29249 leader_election.cc:302] T f0188e48270242949913034b04dbd3b0 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:30.394641 31624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0188e48270242949913034b04dbd3b0" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:30.394765 31624 raft_consensus.cc:2454] T f0188e48270242949913034b04dbd3b0 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:30.395015 31474 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:30.395031 4444 raft_consensus.cc:2781] T f0188e48270242949913034b04dbd3b0 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:30.395082 4444 raft_consensus.cc:482] T f0188e48270242949913034b04dbd3b0 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:30.395116 4444 raft_consensus.cc:3037] T f0188e48270242949913034b04dbd3b0 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:30.396122 4444 raft_consensus.cc:504] T f0188e48270242949913034b04dbd3b0 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:30.396456 4444 leader_election.cc:288] T f0188e48270242949913034b04dbd3b0 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:30.396869 31624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0188e48270242949913034b04dbd3b0" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:30.396896 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0188e48270242949913034b04dbd3b0" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:30.396977 31624 raft_consensus.cc:3037] T f0188e48270242949913034b04dbd3b0 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:30.396991 29577 raft_consensus.cc:3037] T f0188e48270242949913034b04dbd3b0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:30.397267 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9da27cc9c2504b7899da253a88dd1f56" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:30.397378 31625 raft_consensus.cc:2362] T 9da27cc9c2504b7899da253a88dd1f56 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1. Re-sending same reply.
I20240502 20:46:30.397732 6069 leader_election.cc:288] T 9da27cc9c2504b7899da253a88dd1f56 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:30.397784 31624 raft_consensus.cc:2454] T f0188e48270242949913034b04dbd3b0 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:30.397920 29577 raft_consensus.cc:2454] T f0188e48270242949913034b04dbd3b0 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:30.398065 29249 leader_election.cc:302] T f0188e48270242949913034b04dbd3b0 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 315915efcf2b413ca7f9d874338e1b2d; no voters:
I20240502 20:46:30.398237 4444 raft_consensus.cc:2781] T f0188e48270242949913034b04dbd3b0 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:30.398300 4444 raft_consensus.cc:686] T f0188e48270242949913034b04dbd3b0 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:30.398418 4444 consensus_queue.cc:237] T f0188e48270242949913034b04dbd3b0 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:30.401347 31602 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:30.412863 29425 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 20:46:30.416430 31735 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:30.421278 29037 catalog_manager.cc:5484] T f0188e48270242949913034b04dbd3b0 P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:30.422566 31868 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:30.426545 29038 catalog_manager.cc:5484] T 9da27cc9c2504b7899da253a88dd1f56 P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: term changed from 0 to 1, leader changed from <none> to db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:30.437412 29162 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 20:46:30.483593 29574 consensus_queue.cc:237] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:30.484740 31495 raft_consensus.cc:1261] T e7642b18bcbb467187fc8c8b924b79df P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:30.484767 29315 raft_consensus.cc:1261] T e7642b18bcbb467187fc8c8b924b79df P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:30.485158 6054 consensus_queue.cc:1035] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:30.485325 6069 consensus_queue.cc:1035] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20240502 20:46:30.486583 29624 tablet.cc:2369] T 9da27cc9c2504b7899da253a88dd1f56 P db17f26a73f3447b92aa9a947e3a7057: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 20:46:30.487967 6069 raft_consensus.cc:2932] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:30.489012 29315 raft_consensus.cc:2932] T e7642b18bcbb467187fc8c8b924b79df P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:30.489557 31494 raft_consensus.cc:2932] T e7642b18bcbb467187fc8c8b924b79df P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:30.490916 29038 catalog_manager.cc:5484] T e7642b18bcbb467187fc8c8b924b79df P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:30.493901 29425 tablet_service.cc:1514] Processing DeleteTablet for tablet e7642b18bcbb467187fc8c8b924b79df with delete_type TABLET_DATA_TOMBSTONED (TS dbcdc6ac635b44b185c3eba63f66e9d6 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:30.494145 5990 tablet_replica.cc:330] T e7642b18bcbb467187fc8c8b924b79df P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:30.494292 5990 raft_consensus.cc:2229] T e7642b18bcbb467187fc8c8b924b79df P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:30.494426 5990 raft_consensus.cc:2258] T e7642b18bcbb467187fc8c8b924b79df P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:30.495035 5990 ts_tablet_manager.cc:1875] T e7642b18bcbb467187fc8c8b924b79df P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:30.496992 5990 ts_tablet_manager.cc:1888] T e7642b18bcbb467187fc8c8b924b79df P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 20:46:30.497056 5990 log.cc:1196] T e7642b18bcbb467187fc8c8b924b79df P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/e7642b18bcbb467187fc8c8b924b79df
I20240502 20:46:30.497548 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet e7642b18bcbb467187fc8c8b924b79df (table rebalance_test_table_15 [id=02149b4452244d3fbab1649adf739573]) successfully deleted
I20240502 20:46:30.521682 29034 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:30.525240 31624 raft_consensus.cc:1261] T 3d6220f24edd4fdc826a6cccabf599eb P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:30.525607 4444 consensus_queue.cc:1035] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:30.525929 29182 raft_consensus.cc:1261] T 3d6220f24edd4fdc826a6cccabf599eb P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:30.526353 4444 consensus_queue.cc:1035] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:30.529839 6113 tablet.cc:1722] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:30.530159 6113 tablet.cc:1620] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13: MemRowSet was empty: no flush needed.
I20240502 20:46:30.532020 29037 catalog_manager.cc:5576] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751) does not have the latest schema for tablet 3d6220f24edd4fdc826a6cccabf599eb (table rebalancer_dynamic_table [id=a879a41eab5f441da53f51e971c97f42]). Expected version 1 got 0
I20240502 20:46:30.532428 29038 catalog_manager.cc:5576] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267) does not have the latest schema for tablet 3d6220f24edd4fdc826a6cccabf599eb (table rebalancer_dynamic_table [id=a879a41eab5f441da53f51e971c97f42]). Expected version 1 got 0
I20240502 20:46:30.534125 29038 catalog_manager.cc:5576] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751) does not have the latest schema for tablet 3d6220f24edd4fdc826a6cccabf599eb (table rebalancer_dynamic_table [id=a879a41eab5f441da53f51e971c97f42]). Expected version 1 got 0
I20240502 20:46:30.535754 29038 catalog_manager.cc:5576] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267) does not have the latest schema for tablet 3d6220f24edd4fdc826a6cccabf599eb (table rebalancer_dynamic_table [id=a879a41eab5f441da53f51e971c97f42]). Expected version 1 got 0
I20240502 20:46:30.538599 6115 tablet.cc:1722] T 3d6220f24edd4fdc826a6cccabf599eb P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:30.538909 29038 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=a879a41eab5f441da53f51e971c97f42] alter complete (version 1)
I20240502 20:46:30.538933 6115 tablet.cc:1620] T 3d6220f24edd4fdc826a6cccabf599eb P 7796ea3fba294d17a34386873969431d: MemRowSet was empty: no flush needed.
I20240502 20:46:30.539247 6114 tablet.cc:1722] T 3d6220f24edd4fdc826a6cccabf599eb P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:30.539517 6114 tablet.cc:1620] T 3d6220f24edd4fdc826a6cccabf599eb P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:30.614852 29162 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 20:46:30.617780 29294 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:30.620708 31474 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:30.626260 29425 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 20:46:30.628050 31735 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:30.632949 29556 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:30.634083 31602 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:30.640377 31868 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:30.641525 29034 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:30.649122 6113 tablet.cc:1722] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:30.649415 6113 tablet.cc:1620] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13: MemRowSet was empty: no flush needed.
I20240502 20:46:30.649384 6114 tablet.cc:1722] T 3d6220f24edd4fdc826a6cccabf599eb P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:30.649667 6114 tablet.cc:1620] T 3d6220f24edd4fdc826a6cccabf599eb P 315915efcf2b413ca7f9d874338e1b2d: MemRowSet was empty: no flush needed.
I20240502 20:46:30.659034 29041 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=a879a41eab5f441da53f51e971c97f42] alter complete (version 2)
I20240502 20:46:30.661741 6115 tablet.cc:1722] T 3d6220f24edd4fdc826a6cccabf599eb P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:30.662024 6115 tablet.cc:1620] T 3d6220f24edd4fdc826a6cccabf599eb P 7796ea3fba294d17a34386873969431d: MemRowSet was empty: no flush needed.
I20240502 20:46:30.719020 29182 consensus_queue.cc:237] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:30.720296 31755 raft_consensus.cc:1261] T 035762cba9014632ad520f4777544641 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:30.720337 29446 raft_consensus.cc:1261] T 035762cba9014632ad520f4777544641 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:30.720641 5655 consensus_queue.cc:1035] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:30.720834 5656 consensus_queue.cc:1035] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:30.722560 5655 raft_consensus.cc:2932] T 035762cba9014632ad520f4777544641 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:30.722978 31755 raft_consensus.cc:2932] T 035762cba9014632ad520f4777544641 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:30.723889 29446 raft_consensus.cc:2932] T 035762cba9014632ad520f4777544641 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:30.726132 29034 catalog_manager.cc:5484] T 035762cba9014632ad520f4777544641 P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: config changed from index 4 to 5, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) evicted. New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:30.728765 29294 tablet_service.cc:1514] Processing DeleteTablet for tablet 035762cba9014632ad520f4777544641 with delete_type TABLET_DATA_TOMBSTONED (TS 09258ac095664fa99caf0707262cde13 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:30.729317 6170 tablet_replica.cc:330] T 035762cba9014632ad520f4777544641 P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:30.729525 6170 raft_consensus.cc:2229] T 035762cba9014632ad520f4777544641 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:30.729693 6170 raft_consensus.cc:2258] T 035762cba9014632ad520f4777544641 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:30.730129 6170 ts_tablet_manager.cc:1875] T 035762cba9014632ad520f4777544641 P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:30.732050 6170 ts_tablet_manager.cc:1888] T 035762cba9014632ad520f4777544641 P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 20:46:30.732115 6170 log.cc:1196] T 035762cba9014632ad520f4777544641 P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/035762cba9014632ad520f4777544641
I20240502 20:46:30.732532 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet 035762cba9014632ad520f4777544641 (table rebalance_test_table_15 [id=02149b4452244d3fbab1649adf739573]) successfully deleted
I20240502 20:46:30.771373 29034 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:30.776981 6113 tablet.cc:1722] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:30.779781 6115 tablet.cc:1722] T 3d6220f24edd4fdc826a6cccabf599eb P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:30.780586 6114 tablet.cc:1722] T 3d6220f24edd4fdc826a6cccabf599eb P 315915efcf2b413ca7f9d874338e1b2d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:30.784524 29040 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=a879a41eab5f441da53f51e971c97f42] alter complete (version 3)
I20240502 20:46:30.814548 29446 consensus_queue.cc:237] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:30.815919 29574 raft_consensus.cc:1261] T db9c5ce233da4b1a9ce70ae04aef7658 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 20:46:30.815970 29315 raft_consensus.cc:1261] T db9c5ce233da4b1a9ce70ae04aef7658 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 20:46:30.816192 29380 consensus_peers.cc:485] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: db9c5ce233da4b1a9ce70ae04aef7658. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:30.816359 5471 consensus_queue.cc:1035] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:30.816560 6234 consensus_queue.cc:1035] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:30.818554 5471 raft_consensus.cc:2932] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:30.818923 29574 raft_consensus.cc:2932] T db9c5ce233da4b1a9ce70ae04aef7658 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:30.820470 29315 raft_consensus.cc:2932] T db9c5ce233da4b1a9ce70ae04aef7658 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:30.821359 29034 catalog_manager.cc:5484] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index -1 to 4, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:30.831593 5843 consensus_queue.cc:1035] T f0188e48270242949913034b04dbd3b0 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:30.834323 5843 consensus_queue.cc:1035] T f0188e48270242949913034b04dbd3b0 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:30.870923 6234 consensus_queue.cc:1035] T aca63a2154fc4f688fd3a254b572de54 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:30.874879 6237 consensus_queue.cc:1035] T aca63a2154fc4f688fd3a254b572de54 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:30.888244 29041 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:30.888444 29041 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:30.891242 29041 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 3d6220f24edd4fdc826a6cccabf599eb
I20240502 20:46:30.892235 29162 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d6220f24edd4fdc826a6cccabf599eb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:30 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:30.892545 6288 tablet_replica.cc:330] T 3d6220f24edd4fdc826a6cccabf599eb P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:30.892647 31602 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d6220f24edd4fdc826a6cccabf599eb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:30 UTC) from {username='slave'} at 127.0.0.1:58346
I20240502 20:46:30.892896 6288 raft_consensus.cc:2229] T 3d6220f24edd4fdc826a6cccabf599eb P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:30.893078 6288 raft_consensus.cc:2258] T 3d6220f24edd4fdc826a6cccabf599eb P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:30.893155 29294 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d6220f24edd4fdc826a6cccabf599eb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:30 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:30.893492 6170 tablet_replica.cc:330] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:30.893610 6170 raft_consensus.cc:2229] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:30.893803 6170 raft_consensus.cc:2258] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:30.894505 6170 ts_tablet_manager.cc:1875] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:30.894585 6288 ts_tablet_manager.cc:1875] T 3d6220f24edd4fdc826a6cccabf599eb P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:30.896643 6170 ts_tablet_manager.cc:1888] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:30.896643 6288 ts_tablet_manager.cc:1888] T 3d6220f24edd4fdc826a6cccabf599eb P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:30.896706 6288 log.cc:1196] T 3d6220f24edd4fdc826a6cccabf599eb P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/3d6220f24edd4fdc826a6cccabf599eb
I20240502 20:46:30.896706 6170 log.cc:1196] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/3d6220f24edd4fdc826a6cccabf599eb
I20240502 20:46:30.896955 6170 ts_tablet_manager.cc:1909] T 3d6220f24edd4fdc826a6cccabf599eb P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:30.896968 6288 ts_tablet_manager.cc:1909] T 3d6220f24edd4fdc826a6cccabf599eb P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:30.897696 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet 3d6220f24edd4fdc826a6cccabf599eb (table rebalancer_dynamic_table_new_name [id=a879a41eab5f441da53f51e971c97f42]) successfully deleted
I20240502 20:46:30.898684 6289 tablet_replica.cc:330] T 3d6220f24edd4fdc826a6cccabf599eb P 315915efcf2b413ca7f9d874338e1b2d: stopping tablet replica
I20240502 20:46:30.898830 6289 raft_consensus.cc:2229] T 3d6220f24edd4fdc826a6cccabf599eb P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:30.898981 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 3d6220f24edd4fdc826a6cccabf599eb (table rebalancer_dynamic_table_new_name [id=a879a41eab5f441da53f51e971c97f42]) successfully deleted
I20240502 20:46:30.899008 6289 raft_consensus.cc:2258] T 3d6220f24edd4fdc826a6cccabf599eb P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:30.899998 6289 ts_tablet_manager.cc:1875] T 3d6220f24edd4fdc826a6cccabf599eb P 315915efcf2b413ca7f9d874338e1b2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:30.902802 6289 ts_tablet_manager.cc:1888] T 3d6220f24edd4fdc826a6cccabf599eb P 315915efcf2b413ca7f9d874338e1b2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:30.902925 6289 log.cc:1196] T 3d6220f24edd4fdc826a6cccabf599eb P 315915efcf2b413ca7f9d874338e1b2d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal/wals/3d6220f24edd4fdc826a6cccabf599eb
I20240502 20:46:30.903256 6289 ts_tablet_manager.cc:1909] T 3d6220f24edd4fdc826a6cccabf599eb P 315915efcf2b413ca7f9d874338e1b2d: Deleting consensus metadata
I20240502 20:46:30.904286 29027 catalog_manager.cc:4830] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): tablet 3d6220f24edd4fdc826a6cccabf599eb (table rebalancer_dynamic_table_new_name [id=a879a41eab5f441da53f51e971c97f42]) successfully deleted
I20240502 20:46:30.911640 6054 consensus_queue.cc:1035] T 9da27cc9c2504b7899da253a88dd1f56 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:30.920876 5346 consensus_queue.cc:1035] T 9da27cc9c2504b7899da253a88dd1f56 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:30.935900 29034 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_48"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:30.942329 29556 tablet_service.cc:1467] Processing CreateTablet for tablet 378b1151c07648919504bacdcb20a087 (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=eeacac38d5ca43508e09117e5ec5cba6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:30.942651 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 378b1151c07648919504bacdcb20a087. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:30.943647 31868 tablet_service.cc:1467] Processing CreateTablet for tablet 61a0fb22371446f7b33c22ab54262895 (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=eeacac38d5ca43508e09117e5ec5cba6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:30.943643 31602 tablet_service.cc:1467] Processing CreateTablet for tablet 378b1151c07648919504bacdcb20a087 (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=eeacac38d5ca43508e09117e5ec5cba6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:30.943874 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61a0fb22371446f7b33c22ab54262895. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:30.943902 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 378b1151c07648919504bacdcb20a087. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:30.945096 29557 tablet_service.cc:1467] Processing CreateTablet for tablet f9356fb558e847f2a0671d763885504a (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=eeacac38d5ca43508e09117e5ec5cba6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:30.945166 6303 tablet_bootstrap.cc:492] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:30.945305 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9356fb558e847f2a0671d763885504a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:30.946277 6303 tablet_bootstrap.cc:654] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:30.947057 29294 tablet_service.cc:1467] Processing CreateTablet for tablet f9356fb558e847f2a0671d763885504a (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=eeacac38d5ca43508e09117e5ec5cba6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:30.947304 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9356fb558e847f2a0671d763885504a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:30.948040 31735 tablet_service.cc:1467] Processing CreateTablet for tablet f9356fb558e847f2a0671d763885504a (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=eeacac38d5ca43508e09117e5ec5cba6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:30.948292 31735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9356fb558e847f2a0671d763885504a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:30.949585 6307 tablet_bootstrap.cc:492] T f9356fb558e847f2a0671d763885504a P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:30.950729 31736 tablet_service.cc:1467] Processing CreateTablet for tablet 61a0fb22371446f7b33c22ab54262895 (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=eeacac38d5ca43508e09117e5ec5cba6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:30.950951 31736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61a0fb22371446f7b33c22ab54262895. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:30.951658 6307 tablet_bootstrap.cc:654] T f9356fb558e847f2a0671d763885504a P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:30.952505 6304 tablet_bootstrap.cc:492] T 61a0fb22371446f7b33c22ab54262895 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:30.953220 6308 tablet_bootstrap.cc:492] T f9356fb558e847f2a0671d763885504a P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:30.953549 29162 tablet_service.cc:1467] Processing CreateTablet for tablet 61a0fb22371446f7b33c22ab54262895 (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=eeacac38d5ca43508e09117e5ec5cba6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:30.953786 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61a0fb22371446f7b33c22ab54262895. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:30.953862 29425 tablet_service.cc:1467] Processing CreateTablet for tablet 378b1151c07648919504bacdcb20a087 (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=eeacac38d5ca43508e09117e5ec5cba6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:30.954128 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 378b1151c07648919504bacdcb20a087. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:30.955626 6308 tablet_bootstrap.cc:654] T f9356fb558e847f2a0671d763885504a P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:30.956542 6304 tablet_bootstrap.cc:654] T 61a0fb22371446f7b33c22ab54262895 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:30.957433 6310 tablet_bootstrap.cc:492] T 61a0fb22371446f7b33c22ab54262895 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:30.958410 6310 tablet_bootstrap.cc:654] T 61a0fb22371446f7b33c22ab54262895 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:30.960714 6305 tablet_bootstrap.cc:492] T 378b1151c07648919504bacdcb20a087 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:30.959856 6310 tablet_bootstrap.cc:492] T 61a0fb22371446f7b33c22ab54262895 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:30.961802 6305 tablet_bootstrap.cc:654] T 378b1151c07648919504bacdcb20a087 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:30.965816 6311 tablet_bootstrap.cc:492] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:30.966538 29294 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:30.966888 6311 tablet_bootstrap.cc:654] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:30.967995 6304 tablet_bootstrap.cc:492] T 61a0fb22371446f7b33c22ab54262895 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:30.968083 6304 ts_tablet_manager.cc:1367] T 61a0fb22371446f7b33c22ab54262895 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240502 20:46:30.968600 6304 raft_consensus.cc:348] T 61a0fb22371446f7b33c22ab54262895 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:30.968703 6304 raft_consensus.cc:374] T 61a0fb22371446f7b33c22ab54262895 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:30.968737 6304 raft_consensus.cc:729] T 61a0fb22371446f7b33c22ab54262895 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:30.968844 6304 consensus_queue.cc:260] T 61a0fb22371446f7b33c22ab54262895 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:30.961829 6310 ts_tablet_manager.cc:1367] T 61a0fb22371446f7b33c22ab54262895 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:30.971561 6310 raft_consensus.cc:348] T 61a0fb22371446f7b33c22ab54262895 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:30.971719 6310 raft_consensus.cc:374] T 61a0fb22371446f7b33c22ab54262895 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:30.971791 6304 ts_tablet_manager.cc:1398] T 61a0fb22371446f7b33c22ab54262895 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:30.971804 6310 raft_consensus.cc:729] T 61a0fb22371446f7b33c22ab54262895 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:30.971997 6310 consensus_queue.cc:260] T 61a0fb22371446f7b33c22ab54262895 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:30.972429 6310 ts_tablet_manager.cc:1398] T 61a0fb22371446f7b33c22ab54262895 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:30.972509 6308 tablet_bootstrap.cc:492] T f9356fb558e847f2a0671d763885504a P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:30.972594 6308 ts_tablet_manager.cc:1367] T f9356fb558e847f2a0671d763885504a P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20240502 20:46:30.972994 6308 raft_consensus.cc:348] T f9356fb558e847f2a0671d763885504a P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:30.973089 6308 raft_consensus.cc:374] T f9356fb558e847f2a0671d763885504a P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:30.973120 6308 raft_consensus.cc:729] T f9356fb558e847f2a0671d763885504a P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:30.973228 6308 consensus_queue.cc:260] T f9356fb558e847f2a0671d763885504a P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:30.973490 6308 ts_tablet_manager.cc:1398] T f9356fb558e847f2a0671d763885504a P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:30.973601 6308 tablet_bootstrap.cc:492] T 61a0fb22371446f7b33c22ab54262895 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:30.974335 6308 tablet_bootstrap.cc:654] T 61a0fb22371446f7b33c22ab54262895 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:30.976146 6308 tablet_bootstrap.cc:492] T 61a0fb22371446f7b33c22ab54262895 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:30.976220 6308 ts_tablet_manager.cc:1367] T 61a0fb22371446f7b33c22ab54262895 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 20:46:30.976663 6308 raft_consensus.cc:348] T 61a0fb22371446f7b33c22ab54262895 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:30.976749 6308 raft_consensus.cc:374] T 61a0fb22371446f7b33c22ab54262895 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:30.976776 6308 raft_consensus.cc:729] T 61a0fb22371446f7b33c22ab54262895 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:30.976878 6308 consensus_queue.cc:260] T 61a0fb22371446f7b33c22ab54262895 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:30.977146 6308 ts_tablet_manager.cc:1398] T 61a0fb22371446f7b33c22ab54262895 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:30.977386 6305 tablet_bootstrap.cc:492] T 378b1151c07648919504bacdcb20a087 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:30.977470 6305 ts_tablet_manager.cc:1367] T 378b1151c07648919504bacdcb20a087 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 20:46:30.977995 6305 raft_consensus.cc:348] T 378b1151c07648919504bacdcb20a087 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:30.978094 6305 raft_consensus.cc:374] T 378b1151c07648919504bacdcb20a087 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:30.978127 6305 raft_consensus.cc:729] T 378b1151c07648919504bacdcb20a087 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:30.978231 6305 consensus_queue.cc:260] T 378b1151c07648919504bacdcb20a087 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:30.978528 6305 ts_tablet_manager.cc:1398] T 378b1151c07648919504bacdcb20a087 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:30.982087 29557 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:30.982425 31602 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 20:46:30.988863 29425 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 20:46:30.993798 29034 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:30.997839 31474 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:31.003084 29425 tablet_service.cc:1467] Processing CreateTablet for tablet 82922919c9cd4604973a9f9d1e2fc53a (DEFAULT_TABLE table=rebalancer_dynamic_table [id=e5bdf51ceea74bacba1f0406a53d0955]), partition=
I20240502 20:46:31.003196 29162 tablet_service.cc:1467] Processing CreateTablet for tablet 82922919c9cd4604973a9f9d1e2fc53a (DEFAULT_TABLE table=rebalancer_dynamic_table [id=e5bdf51ceea74bacba1f0406a53d0955]), partition=
I20240502 20:46:31.003301 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82922919c9cd4604973a9f9d1e2fc53a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:31.003405 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82922919c9cd4604973a9f9d1e2fc53a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:31.005383 29294 tablet_service.cc:1467] Processing CreateTablet for tablet 82922919c9cd4604973a9f9d1e2fc53a (DEFAULT_TABLE table=rebalancer_dynamic_table [id=e5bdf51ceea74bacba1f0406a53d0955]), partition=
I20240502 20:46:31.005568 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82922919c9cd4604973a9f9d1e2fc53a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:31.005990 31868 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:31.017127 29162 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 20:46:31.026989 6303 tablet_bootstrap.cc:492] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:31.027099 6303 ts_tablet_manager.cc:1367] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.082s user 0.001s sys 0.001s
I20240502 20:46:31.027520 6303 raft_consensus.cc:348] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:31.027622 6303 raft_consensus.cc:374] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:31.027648 6303 raft_consensus.cc:729] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:31.027767 6303 consensus_queue.cc:260] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:31.028020 6303 ts_tablet_manager.cc:1398] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:31.028189 6303 tablet_bootstrap.cc:492] T f9356fb558e847f2a0671d763885504a P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:31.029067 6303 tablet_bootstrap.cc:654] T f9356fb558e847f2a0671d763885504a P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:31.031280 6303 tablet_bootstrap.cc:492] T f9356fb558e847f2a0671d763885504a P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:31.031409 6303 ts_tablet_manager.cc:1367] T f9356fb558e847f2a0671d763885504a P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 20:46:31.032325 6303 raft_consensus.cc:348] T f9356fb558e847f2a0671d763885504a P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:31.033269 6303 raft_consensus.cc:374] T f9356fb558e847f2a0671d763885504a P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:31.033448 6303 raft_consensus.cc:729] T f9356fb558e847f2a0671d763885504a P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:31.033627 6303 consensus_queue.cc:260] T f9356fb558e847f2a0671d763885504a P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:31.034035 6303 ts_tablet_manager.cc:1398] T f9356fb558e847f2a0671d763885504a P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:31.035223 6307 tablet_bootstrap.cc:492] T f9356fb558e847f2a0671d763885504a P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:31.035566 6307 ts_tablet_manager.cc:1367] T f9356fb558e847f2a0671d763885504a P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.086s user 0.002s sys 0.000s
I20240502 20:46:31.035921 31736 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:31.037024 6307 raft_consensus.cc:348] T f9356fb558e847f2a0671d763885504a P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:31.037194 6307 raft_consensus.cc:374] T f9356fb558e847f2a0671d763885504a P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:31.037263 6307 raft_consensus.cc:729] T f9356fb558e847f2a0671d763885504a P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:31.037406 6307 consensus_queue.cc:260] T f9356fb558e847f2a0671d763885504a P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:31.038605 6307 ts_tablet_manager.cc:1398] T f9356fb558e847f2a0671d763885504a P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:31.042773 6307 tablet_bootstrap.cc:492] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:31.044016 6307 tablet_bootstrap.cc:654] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:31.053390 4329 raft_consensus.cc:482] T 61a0fb22371446f7b33c22ab54262895 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:31.053483 4329 raft_consensus.cc:504] T 61a0fb22371446f7b33c22ab54262895 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:31.053978 4329 leader_election.cc:288] T 61a0fb22371446f7b33c22ab54262895 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:31.054265 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61a0fb22371446f7b33c22ab54262895" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:31.054397 31756 raft_consensus.cc:2454] T 61a0fb22371446f7b33c22ab54262895 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:31.054399 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61a0fb22371446f7b33c22ab54262895" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:31.054522 29182 raft_consensus.cc:2454] T 61a0fb22371446f7b33c22ab54262895 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:31.054803 31824 leader_election.cc:302] T 61a0fb22371446f7b33c22ab54262895 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:31.055085 4329 raft_consensus.cc:2781] T 61a0fb22371446f7b33c22ab54262895 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:31.055136 4329 raft_consensus.cc:482] T 61a0fb22371446f7b33c22ab54262895 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:31.055178 4329 raft_consensus.cc:3037] T 61a0fb22371446f7b33c22ab54262895 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:31.055987 4329 raft_consensus.cc:504] T 61a0fb22371446f7b33c22ab54262895 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:31.056264 4329 leader_election.cc:288] T 61a0fb22371446f7b33c22ab54262895 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:31.056607 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61a0fb22371446f7b33c22ab54262895" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:31.056718 29182 raft_consensus.cc:3037] T 61a0fb22371446f7b33c22ab54262895 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:31.057790 29182 raft_consensus.cc:2454] T 61a0fb22371446f7b33c22ab54262895 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:31.058127 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61a0fb22371446f7b33c22ab54262895" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:31.058202 31824 leader_election.cc:302] T 61a0fb22371446f7b33c22ab54262895 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:31.058225 31756 raft_consensus.cc:3037] T 61a0fb22371446f7b33c22ab54262895 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:31.058763 4329 raft_consensus.cc:2781] T 61a0fb22371446f7b33c22ab54262895 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:31.058952 4329 raft_consensus.cc:686] T 61a0fb22371446f7b33c22ab54262895 P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Becoming Leader. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Running, Role: LEADER
I20240502 20:46:31.059083 4329 consensus_queue.cc:237] T 61a0fb22371446f7b33c22ab54262895 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:31.059212 31756 raft_consensus.cc:2454] T 61a0fb22371446f7b33c22ab54262895 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:31.061928 6310 tablet_bootstrap.cc:492] T 82922919c9cd4604973a9f9d1e2fc53a P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:31.063228 29034 catalog_manager.cc:5484] T 61a0fb22371446f7b33c22ab54262895 P e6b60803d0764d228c863e2da65e3691 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6b60803d0764d228c863e2da65e3691 (127.22.45.72). New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:31.063558 6310 tablet_bootstrap.cc:654] T 82922919c9cd4604973a9f9d1e2fc53a P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:31.074959 6310 tablet_bootstrap.cc:492] T 82922919c9cd4604973a9f9d1e2fc53a P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:31.075459 6307 tablet_bootstrap.cc:492] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:31.075603 6307 ts_tablet_manager.cc:1367] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.000s
I20240502 20:46:31.075634 6311 tablet_bootstrap.cc:492] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:31.075745 6311 ts_tablet_manager.cc:1367] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.110s user 0.002s sys 0.000s
I20240502 20:46:31.076236 6307 raft_consensus.cc:348] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:31.076314 6311 raft_consensus.cc:348] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:31.076401 6307 raft_consensus.cc:374] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:31.076458 6311 raft_consensus.cc:374] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:31.076499 6311 raft_consensus.cc:729] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:31.076529 6307 raft_consensus.cc:729] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:31.076622 6311 consensus_queue.cc:260] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:31.076756 6307 consensus_queue.cc:260] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:31.076915 6311 ts_tablet_manager.cc:1398] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:31.077093 6311 tablet_bootstrap.cc:492] T 82922919c9cd4604973a9f9d1e2fc53a P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:31.077203 6307 ts_tablet_manager.cc:1398] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:31.077895 6311 tablet_bootstrap.cc:654] T 82922919c9cd4604973a9f9d1e2fc53a P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:31.078915 6310 ts_tablet_manager.cc:1367] T 82922919c9cd4604973a9f9d1e2fc53a P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.001s
I20240502 20:46:31.080219 6310 raft_consensus.cc:348] T 82922919c9cd4604973a9f9d1e2fc53a P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:31.080384 6310 raft_consensus.cc:374] T 82922919c9cd4604973a9f9d1e2fc53a P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:31.080451 6310 raft_consensus.cc:729] T 82922919c9cd4604973a9f9d1e2fc53a P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:31.080611 6310 consensus_queue.cc:260] T 82922919c9cd4604973a9f9d1e2fc53a P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:31.081091 6310 ts_tablet_manager.cc:1398] T 82922919c9cd4604973a9f9d1e2fc53a P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:31.081826 6311 tablet_bootstrap.cc:492] T 82922919c9cd4604973a9f9d1e2fc53a P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:31.081907 6311 ts_tablet_manager.cc:1367] T 82922919c9cd4604973a9f9d1e2fc53a P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 20:46:31.082293 6311 raft_consensus.cc:348] T 82922919c9cd4604973a9f9d1e2fc53a P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:31.082382 6311 raft_consensus.cc:374] T 82922919c9cd4604973a9f9d1e2fc53a P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:31.082410 6311 raft_consensus.cc:729] T 82922919c9cd4604973a9f9d1e2fc53a P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:31.082563 6311 consensus_queue.cc:260] T 82922919c9cd4604973a9f9d1e2fc53a P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:31.082845 6311 ts_tablet_manager.cc:1398] T 82922919c9cd4604973a9f9d1e2fc53a P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:31.104562 5843 raft_consensus.cc:482] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:31.104658 5843 raft_consensus.cc:504] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:31.104789 4444 raft_consensus.cc:482] T f9356fb558e847f2a0671d763885504a P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:31.104851 4444 raft_consensus.cc:504] T f9356fb558e847f2a0671d763885504a P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:31.105077 5843 leader_election.cc:288] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:31.105213 4444 leader_election.cc:288] T f9356fb558e847f2a0671d763885504a P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:31.105355 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82922919c9cd4604973a9f9d1e2fc53a" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:31.105475 29182 raft_consensus.cc:2454] T 82922919c9cd4604973a9f9d1e2fc53a P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:31.105501 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82922919c9cd4604973a9f9d1e2fc53a" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:31.105510 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9356fb558e847f2a0671d763885504a" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:31.105619 29446 raft_consensus.cc:2454] T 82922919c9cd4604973a9f9d1e2fc53a P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:31.105633 31756 raft_consensus.cc:2454] T f9356fb558e847f2a0671d763885504a P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:31.105901 29251 leader_election.cc:302] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:31.105901 29248 leader_election.cc:302] T f9356fb558e847f2a0671d763885504a P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:31.106009 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9356fb558e847f2a0671d763885504a" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:31.106151 29574 raft_consensus.cc:2454] T f9356fb558e847f2a0671d763885504a P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:31.106151 4444 raft_consensus.cc:2781] T f9356fb558e847f2a0671d763885504a P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:31.106211 4444 raft_consensus.cc:482] T f9356fb558e847f2a0671d763885504a P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:31.106249 4444 raft_consensus.cc:3037] T f9356fb558e847f2a0671d763885504a P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:31.107161 4444 raft_consensus.cc:504] T f9356fb558e847f2a0671d763885504a P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:31.107488 4444 leader_election.cc:288] T f9356fb558e847f2a0671d763885504a P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:31.107712 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9356fb558e847f2a0671d763885504a" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:31.107811 29574 raft_consensus.cc:3037] T f9356fb558e847f2a0671d763885504a P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:31.106163 5843 raft_consensus.cc:2781] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:31.107973 5843 raft_consensus.cc:482] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:31.108012 5843 raft_consensus.cc:3037] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:31.108079 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9356fb558e847f2a0671d763885504a" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:31.108176 31756 raft_consensus.cc:3037] T f9356fb558e847f2a0671d763885504a P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:31.108742 29574 raft_consensus.cc:2454] T f9356fb558e847f2a0671d763885504a P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:31.108912 5843 raft_consensus.cc:504] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:31.109095 31756 raft_consensus.cc:2454] T f9356fb558e847f2a0671d763885504a P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:31.109130 29249 leader_election.cc:302] T f9356fb558e847f2a0671d763885504a P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:31.109251 5843 leader_election.cc:288] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:31.109277 4444 raft_consensus.cc:2781] T f9356fb558e847f2a0671d763885504a P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:31.109344 4444 raft_consensus.cc:686] T f9356fb558e847f2a0671d763885504a P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:31.109457 4444 consensus_queue.cc:237] T f9356fb558e847f2a0671d763885504a P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:31.109670 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82922919c9cd4604973a9f9d1e2fc53a" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:31.109777 29446 raft_consensus.cc:3037] T 82922919c9cd4604973a9f9d1e2fc53a P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:31.109933 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82922919c9cd4604973a9f9d1e2fc53a" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:31.110036 29182 raft_consensus.cc:3037] T 82922919c9cd4604973a9f9d1e2fc53a P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:31.110718 29446 raft_consensus.cc:2454] T 82922919c9cd4604973a9f9d1e2fc53a P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:31.110960 29182 raft_consensus.cc:2454] T 82922919c9cd4604973a9f9d1e2fc53a P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:31.111101 29250 leader_election.cc:302] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:31.111644 5843 raft_consensus.cc:2781] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:31.111724 5843 raft_consensus.cc:686] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:31.111828 5843 consensus_queue.cc:237] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:31.112387 29037 catalog_manager.cc:5484] T f9356fb558e847f2a0671d763885504a P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:31.115880 29037 catalog_manager.cc:5484] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:31.232620 29042 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:31.236438 29182 raft_consensus.cc:1261] T 82922919c9cd4604973a9f9d1e2fc53a P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:31.236907 29444 raft_consensus.cc:1261] T 82922919c9cd4604973a9f9d1e2fc53a P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:31.236946 4444 consensus_queue.cc:1035] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:31.237258 4444 consensus_queue.cc:1035] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:31.239297 5865 mvcc.cc:205] Tried to move back new op lower bound from 7023340712900599808 to 7023340712395239424. Current Snapshot: MvccSnapshot[applied={T|T < 7023340712900599808}]
I20240502 20:46:31.239473 6113 tablet.cc:1722] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:31.239781 6113 tablet.cc:1620] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13: MemRowSet was empty: no flush needed.
I20240502 20:46:31.240309 29033 catalog_manager.cc:5576] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267) does not have the latest schema for tablet 82922919c9cd4604973a9f9d1e2fc53a (table rebalancer_dynamic_table [id=e5bdf51ceea74bacba1f0406a53d0955]). Expected version 1 got 0
I20240502 20:46:31.241580 29033 catalog_manager.cc:5576] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095) does not have the latest schema for tablet 82922919c9cd4604973a9f9d1e2fc53a (table rebalancer_dynamic_table [id=e5bdf51ceea74bacba1f0406a53d0955]). Expected version 1 got 0
I20240502 20:46:31.242759 6115 tablet.cc:1722] T 82922919c9cd4604973a9f9d1e2fc53a P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:31.243037 6115 tablet.cc:1620] T 82922919c9cd4604973a9f9d1e2fc53a P 7796ea3fba294d17a34386873969431d: MemRowSet was empty: no flush needed.
I20240502 20:46:31.244870 29033 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=e5bdf51ceea74bacba1f0406a53d0955] alter complete (version 1)
I20240502 20:46:31.246289 6371 tablet.cc:1722] T 82922919c9cd4604973a9f9d1e2fc53a P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:31.246721 6371 tablet.cc:1620] T 82922919c9cd4604973a9f9d1e2fc53a P dbcdc6ac635b44b185c3eba63f66e9d6: MemRowSet was empty: no flush needed.
I20240502 20:46:31.247776 29042 catalog_manager.cc:5576] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095) does not have the latest schema for tablet 82922919c9cd4604973a9f9d1e2fc53a (table rebalancer_dynamic_table [id=e5bdf51ceea74bacba1f0406a53d0955]). Expected version 1 got 0
I20240502 20:46:31.253474 5804 raft_consensus.cc:482] T 378b1151c07648919504bacdcb20a087 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:31.253572 5804 raft_consensus.cc:504] T 378b1151c07648919504bacdcb20a087 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:31.253988 5804 leader_election.cc:288] T 378b1151c07648919504bacdcb20a087 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:31.254268 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "378b1151c07648919504bacdcb20a087" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:31.254300 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "378b1151c07648919504bacdcb20a087" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:31.254390 29574 raft_consensus.cc:2454] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:31.254405 29444 raft_consensus.cc:2454] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:31.254699 31560 leader_election.cc:302] T 378b1151c07648919504bacdcb20a087 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:31.254851 5804 raft_consensus.cc:2781] T 378b1151c07648919504bacdcb20a087 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:31.254909 5804 raft_consensus.cc:482] T 378b1151c07648919504bacdcb20a087 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:31.254961 5804 raft_consensus.cc:3037] T 378b1151c07648919504bacdcb20a087 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:31.255829 5804 raft_consensus.cc:504] T 378b1151c07648919504bacdcb20a087 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:31.256143 5804 leader_election.cc:288] T 378b1151c07648919504bacdcb20a087 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:31.256412 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "378b1151c07648919504bacdcb20a087" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:31.256522 29574 raft_consensus.cc:3037] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:31.256553 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "378b1151c07648919504bacdcb20a087" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:31.256660 29444 raft_consensus.cc:3037] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:31.257498 29574 raft_consensus.cc:2454] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:31.257498 29444 raft_consensus.cc:2454] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:31.257745 31561 leader_election.cc:302] T 378b1151c07648919504bacdcb20a087 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:31.257899 5804 raft_consensus.cc:2781] T 378b1151c07648919504bacdcb20a087 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:31.258073 5804 raft_consensus.cc:686] T 378b1151c07648919504bacdcb20a087 P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Becoming Leader. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Running, Role: LEADER
I20240502 20:46:31.258204 5804 consensus_queue.cc:237] T 378b1151c07648919504bacdcb20a087 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:31.260181 29036 catalog_manager.cc:5484] T 378b1151c07648919504bacdcb20a087 P 315915efcf2b413ca7f9d874338e1b2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70). New cstate: current_term: 1 leader_uuid: "315915efcf2b413ca7f9d874338e1b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:31.279253 6385 ts_tablet_manager.cc:902] T db9c5ce233da4b1a9ce70ae04aef7658 P 315915efcf2b413ca7f9d874338e1b2d: Initiating tablet copy from peer dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:31.283916 6385 tablet_copy_client.cc:308] T db9c5ce233da4b1a9ce70ae04aef7658 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.67:44095
I20240502 20:46:31.284387 29465 tablet_copy_service.cc:140] P dbcdc6ac635b44b185c3eba63f66e9d6: Received BeginTabletCopySession request for tablet db9c5ce233da4b1a9ce70ae04aef7658 from peer 315915efcf2b413ca7f9d874338e1b2d ({username='slave'} at 127.22.45.70:49775)
I20240502 20:46:31.284476 29465 tablet_copy_service.cc:161] P dbcdc6ac635b44b185c3eba63f66e9d6: Beginning new tablet copy session on tablet db9c5ce233da4b1a9ce70ae04aef7658 from peer 315915efcf2b413ca7f9d874338e1b2d at {username='slave'} at 127.22.45.70:49775: session id = 315915efcf2b413ca7f9d874338e1b2d-db9c5ce233da4b1a9ce70ae04aef7658
I20240502 20:46:31.285393 29465 tablet_copy_source_session.cc:205] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:31.290632 29294 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:31.290773 6385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db9c5ce233da4b1a9ce70ae04aef7658. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:31.291604 31474 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:31.294696 31602 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:31.295397 6385 tablet_copy_client.cc:783] T db9c5ce233da4b1a9ce70ae04aef7658 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:31.295645 6385 tablet_copy_client.cc:647] T db9c5ce233da4b1a9ce70ae04aef7658 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:31.296195 29162 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 20:46:31.299598 6385 tablet_copy_client.cc:523] T db9c5ce233da4b1a9ce70ae04aef7658 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:31.300771 6385 tablet_bootstrap.cc:492] T db9c5ce233da4b1a9ce70ae04aef7658 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:31.302634 31868 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:31.311029 29425 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 20:46:31.318733 29557 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:31.340302 6385 tablet_bootstrap.cc:492] T db9c5ce233da4b1a9ce70ae04aef7658 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:31.340631 6385 tablet_bootstrap.cc:492] T db9c5ce233da4b1a9ce70ae04aef7658 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap complete.
I20240502 20:46:31.340858 6385 ts_tablet_manager.cc:1367] T db9c5ce233da4b1a9ce70ae04aef7658 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.040s user 0.025s sys 0.000s
I20240502 20:46:31.341538 6385 raft_consensus.cc:348] T db9c5ce233da4b1a9ce70ae04aef7658 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:31.341739 6385 raft_consensus.cc:729] T db9c5ce233da4b1a9ce70ae04aef7658 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:31.341928 6385 consensus_queue.cc:260] T db9c5ce233da4b1a9ce70ae04aef7658 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:31.343056 6385 ts_tablet_manager.cc:1398] T db9c5ce233da4b1a9ce70ae04aef7658 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:31.345343 29465 tablet_copy_service.cc:342] P dbcdc6ac635b44b185c3eba63f66e9d6: Request end of tablet copy session 315915efcf2b413ca7f9d874338e1b2d-db9c5ce233da4b1a9ce70ae04aef7658 received from {username='slave'} at 127.22.45.70:49775
I20240502 20:46:31.345414 29465 tablet_copy_service.cc:433] P dbcdc6ac635b44b185c3eba63f66e9d6: ending tablet copy session 315915efcf2b413ca7f9d874338e1b2d-db9c5ce233da4b1a9ce70ae04aef7658 on tablet db9c5ce233da4b1a9ce70ae04aef7658 with peer 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:31.347393 31736 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:31.349407 29036 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:31.356277 6113 tablet.cc:1722] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:31.356576 6113 tablet.cc:1620] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13: MemRowSet was empty: no flush needed.
I20240502 20:46:31.358115 6371 tablet.cc:1722] T 82922919c9cd4604973a9f9d1e2fc53a P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:31.358361 6371 tablet.cc:1620] T 82922919c9cd4604973a9f9d1e2fc53a P dbcdc6ac635b44b185c3eba63f66e9d6: MemRowSet was empty: no flush needed.
I20240502 20:46:31.359768 6115 tablet.cc:1722] T 82922919c9cd4604973a9f9d1e2fc53a P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:31.360028 6115 tablet.cc:1620] T 82922919c9cd4604973a9f9d1e2fc53a P 7796ea3fba294d17a34386873969431d: MemRowSet was empty: no flush needed.
I20240502 20:46:31.367969 29036 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=e5bdf51ceea74bacba1f0406a53d0955] alter complete (version 2)
I20240502 20:46:31.407919 29182 consensus_queue.cc:237] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:31.409000 31886 raft_consensus.cc:1261] T 3943d49acd0341c1a923d900281f1591 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:31.409046 29315 raft_consensus.cc:1261] T 3943d49acd0341c1a923d900281f1591 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:31.409415 6247 consensus_queue.cc:1035] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:31.409624 5655 consensus_queue.cc:1035] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:31.411553 5655 raft_consensus.cc:2932] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:31.411949 31886 raft_consensus.cc:2932] T 3943d49acd0341c1a923d900281f1591 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:31.414777 29036 catalog_manager.cc:5484] T 3943d49acd0341c1a923d900281f1591 P 7796ea3fba294d17a34386873969431d reported cstate change: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:31.414830 29315 raft_consensus.cc:2932] T 3943d49acd0341c1a923d900281f1591 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:31.419194 29425 tablet_service.cc:1514] Processing DeleteTablet for tablet 3943d49acd0341c1a923d900281f1591 with delete_type TABLET_DATA_TOMBSTONED (TS dbcdc6ac635b44b185c3eba63f66e9d6 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:31.419672 6415 tablet_replica.cc:330] T 3943d49acd0341c1a923d900281f1591 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:31.419898 6415 raft_consensus.cc:2229] T 3943d49acd0341c1a923d900281f1591 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:31.420126 6415 raft_consensus.cc:2258] T 3943d49acd0341c1a923d900281f1591 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:31.420631 6415 ts_tablet_manager.cc:1875] T 3943d49acd0341c1a923d900281f1591 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:31.422901 6415 ts_tablet_manager.cc:1888] T 3943d49acd0341c1a923d900281f1591 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 20:46:31.423022 6415 log.cc:1196] T 3943d49acd0341c1a923d900281f1591 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/3943d49acd0341c1a923d900281f1591
I20240502 20:46:31.423483 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet 3943d49acd0341c1a923d900281f1591 (table rebalance_test_table_9 [id=4b510e9cf32a4794a9193f7ebac46f93]) successfully deleted
I20240502 20:46:31.474115 29034 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:31.483927 6113 tablet.cc:1722] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:31.484340 6371 tablet.cc:1722] T 82922919c9cd4604973a9f9d1e2fc53a P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:31.488461 5843 consensus_queue.cc:1035] T f9356fb558e847f2a0671d763885504a P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:31.491068 6115 tablet.cc:1722] T 82922919c9cd4604973a9f9d1e2fc53a P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:31.493309 5843 consensus_queue.cc:1035] T f9356fb558e847f2a0671d763885504a P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:31.494051 29035 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=e5bdf51ceea74bacba1f0406a53d0955] alter complete (version 3)
I20240502 20:46:31.522581 29294 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:31.528301 31474 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:31.530551 29162 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 20:46:31.545802 31602 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:31.546430 29557 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:31.561738 6322 consensus_queue.cc:1035] T 61a0fb22371446f7b33c22ab54262895 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:31.563160 31868 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:31.570842 6320 consensus_queue.cc:1035] T 61a0fb22371446f7b33c22ab54262895 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:31.575140 31736 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:31.582103 29425 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 20:46:31.598124 29041 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:31.598246 29041 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:31.600375 29041 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 82922919c9cd4604973a9f9d1e2fc53a
I20240502 20:46:31.601101 29294 tablet_service.cc:1514] Processing DeleteTablet for tablet 82922919c9cd4604973a9f9d1e2fc53a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:31 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:31.601194 29425 tablet_service.cc:1514] Processing DeleteTablet for tablet 82922919c9cd4604973a9f9d1e2fc53a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:31 UTC) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:31.601370 6415 tablet_replica.cc:330] T 82922919c9cd4604973a9f9d1e2fc53a P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:31.601505 6415 raft_consensus.cc:2229] T 82922919c9cd4604973a9f9d1e2fc53a P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:31.601522 29162 tablet_service.cc:1514] Processing DeleteTablet for tablet 82922919c9cd4604973a9f9d1e2fc53a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:31 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:31.601616 6415 raft_consensus.cc:2258] T 82922919c9cd4604973a9f9d1e2fc53a P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:31.602376 6415 ts_tablet_manager.cc:1875] T 82922919c9cd4604973a9f9d1e2fc53a P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:31.604180 6415 ts_tablet_manager.cc:1888] T 82922919c9cd4604973a9f9d1e2fc53a P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:31.604239 6415 log.cc:1196] T 82922919c9cd4604973a9f9d1e2fc53a P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/82922919c9cd4604973a9f9d1e2fc53a
I20240502 20:46:31.604489 6415 ts_tablet_manager.cc:1909] T 82922919c9cd4604973a9f9d1e2fc53a P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting consensus metadata
I20240502 20:46:31.605072 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet 82922919c9cd4604973a9f9d1e2fc53a (table rebalancer_dynamic_table_new_name [id=e5bdf51ceea74bacba1f0406a53d0955]) successfully deleted
I20240502 20:46:31.605808 6474 tablet_replica.cc:330] T 82922919c9cd4604973a9f9d1e2fc53a P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:31.605978 6474 raft_consensus.cc:2229] T 82922919c9cd4604973a9f9d1e2fc53a P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:31.606125 6474 raft_consensus.cc:2258] T 82922919c9cd4604973a9f9d1e2fc53a P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:31.606372 6473 tablet_replica.cc:330] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:31.606770 6474 ts_tablet_manager.cc:1875] T 82922919c9cd4604973a9f9d1e2fc53a P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:31.609356 6474 ts_tablet_manager.cc:1888] T 82922919c9cd4604973a9f9d1e2fc53a P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:31.609573 6473 raft_consensus.cc:2229] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:31.609825 6473 raft_consensus.cc:2258] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:31.610625 6474 log.cc:1196] T 82922919c9cd4604973a9f9d1e2fc53a P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/82922919c9cd4604973a9f9d1e2fc53a
I20240502 20:46:31.611320 6474 ts_tablet_manager.cc:1909] T 82922919c9cd4604973a9f9d1e2fc53a P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:31.611922 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 82922919c9cd4604973a9f9d1e2fc53a (table rebalancer_dynamic_table_new_name [id=e5bdf51ceea74bacba1f0406a53d0955]) successfully deleted
I20240502 20:46:31.617270 6473 ts_tablet_manager.cc:1875] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:31.622931 6473 ts_tablet_manager.cc:1888] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:31.623005 6473 log.cc:1196] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/82922919c9cd4604973a9f9d1e2fc53a
I20240502 20:46:31.623296 6473 ts_tablet_manager.cc:1909] T 82922919c9cd4604973a9f9d1e2fc53a P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:31.623944 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet 82922919c9cd4604973a9f9d1e2fc53a (table rebalancer_dynamic_table_new_name [id=e5bdf51ceea74bacba1f0406a53d0955]) successfully deleted
I20240502 20:46:31.643168 31495 consensus_queue.cc:237] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:31.644292 29444 raft_consensus.cc:1261] T 0922994ff62d4e648d5132cb08b6a3c8 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:31.644335 29315 raft_consensus.cc:1261] T 0922994ff62d4e648d5132cb08b6a3c8 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:31.644676 4619 consensus_queue.cc:1035] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:31.644855 4975 consensus_queue.cc:1035] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:31.646450 4619 raft_consensus.cc:2932] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:31.647924 29444 raft_consensus.cc:2932] T 0922994ff62d4e648d5132cb08b6a3c8 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:31.648499 29315 raft_consensus.cc:2932] T 0922994ff62d4e648d5132cb08b6a3c8 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:31.649159 29041 catalog_manager.cc:5484] T 0922994ff62d4e648d5132cb08b6a3c8 P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: config changed from index 2 to 3, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) evicted. New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:31.652160 31868 tablet_service.cc:1514] Processing DeleteTablet for tablet 0922994ff62d4e648d5132cb08b6a3c8 with delete_type TABLET_DATA_TOMBSTONED (TS e6b60803d0764d228c863e2da65e3691 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:31.652501 6493 tablet_replica.cc:330] T 0922994ff62d4e648d5132cb08b6a3c8 P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:31.652660 6493 raft_consensus.cc:2229] T 0922994ff62d4e648d5132cb08b6a3c8 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:31.652827 6493 raft_consensus.cc:2258] T 0922994ff62d4e648d5132cb08b6a3c8 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:31.653610 6493 ts_tablet_manager.cc:1875] T 0922994ff62d4e648d5132cb08b6a3c8 P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:31.655476 6493 ts_tablet_manager.cc:1888] T 0922994ff62d4e648d5132cb08b6a3c8 P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 20:46:31.655542 6493 log.cc:1196] T 0922994ff62d4e648d5132cb08b6a3c8 P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/0922994ff62d4e648d5132cb08b6a3c8
I20240502 20:46:31.656220 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet 0922994ff62d4e648d5132cb08b6a3c8 (table rebalancer_extra_table_8 [id=2005c6c66aec4f4d823774ee2b67a575]) successfully deleted
I20240502 20:46:31.702124 29042 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:31.707993 29425 tablet_service.cc:1467] Processing CreateTablet for tablet 8c95bdf553804cd6bab1ecdcf2158faf (DEFAULT_TABLE table=rebalancer_dynamic_table [id=52f922a238a54a2c8224d3fb855ed6bf]), partition=
I20240502 20:46:31.708253 29294 tablet_service.cc:1467] Processing CreateTablet for tablet 8c95bdf553804cd6bab1ecdcf2158faf (DEFAULT_TABLE table=rebalancer_dynamic_table [id=52f922a238a54a2c8224d3fb855ed6bf]), partition=
I20240502 20:46:31.708395 31868 tablet_service.cc:1467] Processing CreateTablet for tablet 8c95bdf553804cd6bab1ecdcf2158faf (DEFAULT_TABLE table=rebalancer_dynamic_table [id=52f922a238a54a2c8224d3fb855ed6bf]), partition=
I20240502 20:46:31.708482 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c95bdf553804cd6bab1ecdcf2158faf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:31.708488 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c95bdf553804cd6bab1ecdcf2158faf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:31.708595 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c95bdf553804cd6bab1ecdcf2158faf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:31.710846 6537 tablet_bootstrap.cc:492] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:31.711838 6537 tablet_bootstrap.cc:654] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:31.712595 6536 tablet_bootstrap.cc:492] T 8c95bdf553804cd6bab1ecdcf2158faf P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:31.713819 6538 tablet_bootstrap.cc:492] T 8c95bdf553804cd6bab1ecdcf2158faf P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:31.714054 6537 tablet_bootstrap.cc:492] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:31.714151 6537 ts_tablet_manager.cc:1367] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:31.714672 6537 raft_consensus.cc:348] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:31.715209 6537 raft_consensus.cc:374] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:31.715301 6537 raft_consensus.cc:729] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:31.715504 6537 consensus_queue.cc:260] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:31.715607 6538 tablet_bootstrap.cc:654] T 8c95bdf553804cd6bab1ecdcf2158faf P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:31.715950 6537 ts_tablet_manager.cc:1398] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:31.716955 6536 tablet_bootstrap.cc:654] T 8c95bdf553804cd6bab1ecdcf2158faf P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:31.718111 6536 tablet_bootstrap.cc:492] T 8c95bdf553804cd6bab1ecdcf2158faf P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:31.718238 6536 ts_tablet_manager.cc:1367] T 8c95bdf553804cd6bab1ecdcf2158faf P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 20:46:31.718551 6538 tablet_bootstrap.cc:492] T 8c95bdf553804cd6bab1ecdcf2158faf P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:31.718652 6538 ts_tablet_manager.cc:1367] T 8c95bdf553804cd6bab1ecdcf2158faf P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 20:46:31.719089 6538 raft_consensus.cc:348] T 8c95bdf553804cd6bab1ecdcf2158faf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:31.719285 6538 raft_consensus.cc:374] T 8c95bdf553804cd6bab1ecdcf2158faf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:31.719341 6538 raft_consensus.cc:729] T 8c95bdf553804cd6bab1ecdcf2158faf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:31.719484 6538 consensus_queue.cc:260] T 8c95bdf553804cd6bab1ecdcf2158faf P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:31.719859 6538 ts_tablet_manager.cc:1398] T 8c95bdf553804cd6bab1ecdcf2158faf P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:31.719170 6536 raft_consensus.cc:348] T 8c95bdf553804cd6bab1ecdcf2158faf P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:31.720783 6536 raft_consensus.cc:374] T 8c95bdf553804cd6bab1ecdcf2158faf P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:31.720865 6536 raft_consensus.cc:729] T 8c95bdf553804cd6bab1ecdcf2158faf P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:31.720999 6536 consensus_queue.cc:260] T 8c95bdf553804cd6bab1ecdcf2158faf P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:31.722764 6536 ts_tablet_manager.cc:1398] T 8c95bdf553804cd6bab1ecdcf2158faf P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:31.748051 31625 raft_consensus.cc:1203] T db9c5ce233da4b1a9ce70ae04aef7658 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 20:46:31.749541 6322 raft_consensus.cc:482] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:31.749629 6322 raft_consensus.cc:504] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:31.750082 6322 leader_election.cc:288] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:31.750326 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c95bdf553804cd6bab1ecdcf2158faf" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:31.750448 29444 raft_consensus.cc:2454] T 8c95bdf553804cd6bab1ecdcf2158faf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:31.750602 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c95bdf553804cd6bab1ecdcf2158faf" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:31.750723 29315 raft_consensus.cc:2454] T 8c95bdf553804cd6bab1ecdcf2158faf P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:31.750893 31823 leader_election.cc:302] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dbcdc6ac635b44b185c3eba63f66e9d6, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:31.751164 6322 raft_consensus.cc:2781] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:31.751220 6322 raft_consensus.cc:482] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:31.751245 6322 raft_consensus.cc:3037] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:31.752346 6322 raft_consensus.cc:504] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:31.752678 6322 leader_election.cc:288] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Requested vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:31.753031 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c95bdf553804cd6bab1ecdcf2158faf" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:31.753103 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c95bdf553804cd6bab1ecdcf2158faf" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:31.753140 29444 raft_consensus.cc:3037] T 8c95bdf553804cd6bab1ecdcf2158faf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:31.753198 29315 raft_consensus.cc:3037] T 8c95bdf553804cd6bab1ecdcf2158faf P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:31.754094 29315 raft_consensus.cc:2454] T 8c95bdf553804cd6bab1ecdcf2158faf P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:31.754154 29444 raft_consensus.cc:2454] T 8c95bdf553804cd6bab1ecdcf2158faf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:31.754338 31823 leader_election.cc:302] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:31.754532 6322 raft_consensus.cc:2781] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:31.754618 6322 raft_consensus.cc:686] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Becoming Leader. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Running, Role: LEADER
I20240502 20:46:31.754740 6322 consensus_queue.cc:237] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:31.759588 29041 catalog_manager.cc:5484] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6b60803d0764d228c863e2da65e3691 (127.22.45.72). New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:31.787357 29294 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:31.793669 31474 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:31.798118 31602 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:31.800674 31736 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:31.802778 31868 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:31.806644 29162 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 20:46:31.819897 29425 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 20:46:31.827013 29041 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_50"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:31.829671 5804 consensus_queue.cc:1035] T 378b1151c07648919504bacdcb20a087 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:31.834870 31474 tablet_service.cc:1467] Processing CreateTablet for tablet af74e9afe64347168b6e28eac41e9860 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=d24483eb514346a18278167baac902b1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:31.834926 31475 tablet_service.cc:1467] Processing CreateTablet for tablet 6e80723ee7624d698e9e0a0bd494e1f0 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=d24483eb514346a18278167baac902b1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:31.835162 31474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af74e9afe64347168b6e28eac41e9860. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:31.835672 29294 tablet_service.cc:1467] Processing CreateTablet for tablet af74e9afe64347168b6e28eac41e9860 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=d24483eb514346a18278167baac902b1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:31.835922 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af74e9afe64347168b6e28eac41e9860. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:31.837514 31735 tablet_service.cc:1467] Processing CreateTablet for tablet 9d3d4caa84c74e4a960f911c1557f8ce (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=d24483eb514346a18278167baac902b1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:31.837754 31735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d3d4caa84c74e4a960f911c1557f8ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:31.837811 31736 tablet_service.cc:1467] Processing CreateTablet for tablet af74e9afe64347168b6e28eac41e9860 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=d24483eb514346a18278167baac902b1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:31.838001 31736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af74e9afe64347168b6e28eac41e9860. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:31.838069 29162 tablet_service.cc:1467] Processing CreateTablet for tablet 9d3d4caa84c74e4a960f911c1557f8ce (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=d24483eb514346a18278167baac902b1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:31.838297 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d3d4caa84c74e4a960f911c1557f8ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:31.839949 31868 tablet_service.cc:1467] Processing CreateTablet for tablet 6e80723ee7624d698e9e0a0bd494e1f0 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=d24483eb514346a18278167baac902b1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:31.840207 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e80723ee7624d698e9e0a0bd494e1f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:31.840349 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e80723ee7624d698e9e0a0bd494e1f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:31.840358 6536 tablet_bootstrap.cc:492] T af74e9afe64347168b6e28eac41e9860 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:31.841264 6536 tablet_bootstrap.cc:654] T af74e9afe64347168b6e28eac41e9860 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:31.842239 6537 tablet_bootstrap.cc:492] T 6e80723ee7624d698e9e0a0bd494e1f0 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:31.843097 6537 tablet_bootstrap.cc:654] T 6e80723ee7624d698e9e0a0bd494e1f0 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:31.844929 29557 tablet_service.cc:1467] Processing CreateTablet for tablet 9d3d4caa84c74e4a960f911c1557f8ce (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=d24483eb514346a18278167baac902b1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:31.845178 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d3d4caa84c74e4a960f911c1557f8ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:31.846964 31602 tablet_service.cc:1467] Processing CreateTablet for tablet 6e80723ee7624d698e9e0a0bd494e1f0 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=d24483eb514346a18278167baac902b1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:31.847193 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e80723ee7624d698e9e0a0bd494e1f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:31.851119 5804 consensus_queue.cc:1035] T 378b1151c07648919504bacdcb20a087 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:31.852573 6578 tablet_bootstrap.cc:492] T 9d3d4caa84c74e4a960f911c1557f8ce P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:31.853397 6537 tablet_bootstrap.cc:492] T 6e80723ee7624d698e9e0a0bd494e1f0 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:31.853480 6537 ts_tablet_manager.cc:1367] T 6e80723ee7624d698e9e0a0bd494e1f0 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 20:46:31.853886 6578 tablet_bootstrap.cc:654] T 9d3d4caa84c74e4a960f911c1557f8ce P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:31.853873 6537 raft_consensus.cc:348] T 6e80723ee7624d698e9e0a0bd494e1f0 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:31.853981 6537 raft_consensus.cc:374] T 6e80723ee7624d698e9e0a0bd494e1f0 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:31.854007 6537 raft_consensus.cc:729] T 6e80723ee7624d698e9e0a0bd494e1f0 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:31.854105 6537 consensus_queue.cc:260] T 6e80723ee7624d698e9e0a0bd494e1f0 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:31.854382 6537 ts_tablet_manager.cc:1398] T 6e80723ee7624d698e9e0a0bd494e1f0 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:31.859300 6576 tablet_bootstrap.cc:492] T 9d3d4caa84c74e4a960f911c1557f8ce P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:31.859532 6577 tablet_bootstrap.cc:492] T af74e9afe64347168b6e28eac41e9860 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:31.859841 6582 tablet_bootstrap.cc:492] T 6e80723ee7624d698e9e0a0bd494e1f0 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:31.860466 6577 tablet_bootstrap.cc:654] T af74e9afe64347168b6e28eac41e9860 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:31.861342 6536 tablet_bootstrap.cc:492] T af74e9afe64347168b6e28eac41e9860 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:31.861433 6536 ts_tablet_manager.cc:1367] T af74e9afe64347168b6e28eac41e9860 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20240502 20:46:31.861909 6576 tablet_bootstrap.cc:654] T 9d3d4caa84c74e4a960f911c1557f8ce P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:31.861860 6536 raft_consensus.cc:348] T af74e9afe64347168b6e28eac41e9860 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:31.861965 6536 raft_consensus.cc:374] T af74e9afe64347168b6e28eac41e9860 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:31.861994 6536 raft_consensus.cc:729] T af74e9afe64347168b6e28eac41e9860 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:31.862111 6536 consensus_queue.cc:260] T af74e9afe64347168b6e28eac41e9860 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:31.862367 6536 ts_tablet_manager.cc:1398] T af74e9afe64347168b6e28eac41e9860 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:31.864209 6582 tablet_bootstrap.cc:654] T 6e80723ee7624d698e9e0a0bd494e1f0 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:31.865239 6576 tablet_bootstrap.cc:492] T 9d3d4caa84c74e4a960f911c1557f8ce P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:31.865365 6576 ts_tablet_manager.cc:1367] T 9d3d4caa84c74e4a960f911c1557f8ce P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 20:46:31.865811 6576 raft_consensus.cc:348] T 9d3d4caa84c74e4a960f911c1557f8ce P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:31.866526 6576 raft_consensus.cc:374] T 9d3d4caa84c74e4a960f911c1557f8ce P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:31.866789 6576 raft_consensus.cc:729] T 9d3d4caa84c74e4a960f911c1557f8ce P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:31.866992 6576 consensus_queue.cc:260] T 9d3d4caa84c74e4a960f911c1557f8ce P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:31.867504 6576 ts_tablet_manager.cc:1398] T 9d3d4caa84c74e4a960f911c1557f8ce P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:31.867700 6576 tablet_bootstrap.cc:492] T af74e9afe64347168b6e28eac41e9860 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:31.869019 29041 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:31.869112 6578 tablet_bootstrap.cc:492] T 9d3d4caa84c74e4a960f911c1557f8ce P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:31.869307 6578 ts_tablet_manager.cc:1367] T 9d3d4caa84c74e4a960f911c1557f8ce P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 20:46:31.873157 6578 raft_consensus.cc:348] T 9d3d4caa84c74e4a960f911c1557f8ce P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:31.873313 6578 raft_consensus.cc:374] T 9d3d4caa84c74e4a960f911c1557f8ce P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:31.873381 6578 raft_consensus.cc:729] T 9d3d4caa84c74e4a960f911c1557f8ce P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:31.873548 6578 consensus_queue.cc:260] T 9d3d4caa84c74e4a960f911c1557f8ce P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:31.873885 6578 ts_tablet_manager.cc:1398] T 9d3d4caa84c74e4a960f911c1557f8ce P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:31.875349 6577 tablet_bootstrap.cc:492] T af74e9afe64347168b6e28eac41e9860 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:31.875481 6577 ts_tablet_manager.cc:1367] T af74e9afe64347168b6e28eac41e9860 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240502 20:46:31.875919 6577 raft_consensus.cc:348] T af74e9afe64347168b6e28eac41e9860 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:31.876077 6577 raft_consensus.cc:374] T af74e9afe64347168b6e28eac41e9860 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:31.876140 6577 raft_consensus.cc:729] T af74e9afe64347168b6e28eac41e9860 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:31.876286 6577 consensus_queue.cc:260] T af74e9afe64347168b6e28eac41e9860 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:31.877372 6577 ts_tablet_manager.cc:1398] T af74e9afe64347168b6e28eac41e9860 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:31.877557 6577 tablet_bootstrap.cc:492] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:31.878501 6577 tablet_bootstrap.cc:654] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:31.878805 29557 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:31.879133 6576 tablet_bootstrap.cc:654] T af74e9afe64347168b6e28eac41e9860 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:31.879130 29315 raft_consensus.cc:1261] T 8c95bdf553804cd6bab1ecdcf2158faf P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:31.879812 6562 consensus_queue.cc:1035] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:31.880209 6577 tablet_bootstrap.cc:492] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:31.880342 6577 ts_tablet_manager.cc:1367] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 20:46:31.880766 6577 raft_consensus.cc:348] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:31.880913 6577 raft_consensus.cc:374] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:31.880964 29444 raft_consensus.cc:1261] T 8c95bdf553804cd6bab1ecdcf2158faf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:31.880990 6577 raft_consensus.cc:729] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:31.881124 6577 consensus_queue.cc:260] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:31.881407 6562 consensus_queue.cc:1035] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:31.881453 6577 ts_tablet_manager.cc:1398] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:31.883230 6593 tablet.cc:1722] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:31.883581 6593 tablet.cc:1620] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:31.883877 29042 catalog_manager.cc:5576] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945) does not have the latest schema for tablet 8c95bdf553804cd6bab1ecdcf2158faf (table rebalancer_dynamic_table [id=52f922a238a54a2c8224d3fb855ed6bf]). Expected version 1 got 0
I20240502 20:46:31.884343 6113 tablet.cc:1722] T 8c95bdf553804cd6bab1ecdcf2158faf P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:31.884593 6113 tablet.cc:1620] T 8c95bdf553804cd6bab1ecdcf2158faf P 09258ac095664fa99caf0707262cde13: MemRowSet was empty: no flush needed.
I20240502 20:46:31.884727 29037 catalog_manager.cc:5576] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095) does not have the latest schema for tablet 8c95bdf553804cd6bab1ecdcf2158faf (table rebalancer_dynamic_table [id=52f922a238a54a2c8224d3fb855ed6bf]). Expected version 1 got 0
I20240502 20:46:31.893587 29041 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=52f922a238a54a2c8224d3fb855ed6bf] alter complete (version 1)
I20240502 20:46:31.895484 6581 tablet_bootstrap.cc:492] T 9d3d4caa84c74e4a960f911c1557f8ce P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:31.896720 6581 tablet_bootstrap.cc:654] T 9d3d4caa84c74e4a960f911c1557f8ce P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:31.899983 6581 tablet_bootstrap.cc:492] T 9d3d4caa84c74e4a960f911c1557f8ce P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:31.900074 6581 ts_tablet_manager.cc:1367] T 9d3d4caa84c74e4a960f911c1557f8ce P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 20:46:31.900482 6581 raft_consensus.cc:348] T 9d3d4caa84c74e4a960f911c1557f8ce P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:31.900584 6581 raft_consensus.cc:374] T 9d3d4caa84c74e4a960f911c1557f8ce P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:31.900655 6581 raft_consensus.cc:729] T 9d3d4caa84c74e4a960f911c1557f8ce P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:31.900772 6581 consensus_queue.cc:260] T 9d3d4caa84c74e4a960f911c1557f8ce P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:31.901160 6581 ts_tablet_manager.cc:1398] T 9d3d4caa84c74e4a960f911c1557f8ce P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:31.901821 6371 tablet.cc:1722] T 8c95bdf553804cd6bab1ecdcf2158faf P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:31.902139 6371 tablet.cc:1620] T 8c95bdf553804cd6bab1ecdcf2158faf P dbcdc6ac635b44b185c3eba63f66e9d6: MemRowSet was empty: no flush needed.
I20240502 20:46:31.903110 6576 tablet_bootstrap.cc:492] T af74e9afe64347168b6e28eac41e9860 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:31.903188 6576 ts_tablet_manager.cc:1367] T af74e9afe64347168b6e28eac41e9860 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.036s user 0.002s sys 0.000s
I20240502 20:46:31.903566 6576 raft_consensus.cc:348] T af74e9afe64347168b6e28eac41e9860 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:31.903657 6576 raft_consensus.cc:374] T af74e9afe64347168b6e28eac41e9860 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:31.903685 6576 raft_consensus.cc:729] T af74e9afe64347168b6e28eac41e9860 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:31.903792 6576 consensus_queue.cc:260] T af74e9afe64347168b6e28eac41e9860 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:31.904021 6576 ts_tablet_manager.cc:1398] T af74e9afe64347168b6e28eac41e9860 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:31.920409 6582 tablet_bootstrap.cc:492] T 6e80723ee7624d698e9e0a0bd494e1f0 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:31.920533 6582 ts_tablet_manager.cc:1367] T 6e80723ee7624d698e9e0a0bd494e1f0 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.061s user 0.002s sys 0.000s
I20240502 20:46:31.921010 6582 raft_consensus.cc:348] T 6e80723ee7624d698e9e0a0bd494e1f0 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:31.921115 6582 raft_consensus.cc:374] T 6e80723ee7624d698e9e0a0bd494e1f0 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:31.921147 6582 raft_consensus.cc:729] T 6e80723ee7624d698e9e0a0bd494e1f0 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:31.921283 6582 consensus_queue.cc:260] T 6e80723ee7624d698e9e0a0bd494e1f0 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:31.921581 6582 ts_tablet_manager.cc:1398] T 6e80723ee7624d698e9e0a0bd494e1f0 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:31.935161 31495 consensus_queue.cc:237] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:31.936302 29315 raft_consensus.cc:1261] T 21ee3d7c6f764d0483b335da40e61efc P 09258ac095664fa99caf0707262cde13 [term 2 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 20:46:31.936302 29175 raft_consensus.cc:1261] T 21ee3d7c6f764d0483b335da40e61efc P 7796ea3fba294d17a34386873969431d [term 2 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 20:46:31.936668 4619 consensus_queue.cc:1035] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 20:46:31.936844 4975 consensus_queue.cc:1035] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 20:46:31.938716 4619 raft_consensus.cc:2932] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:31.940574 29175 raft_consensus.cc:2932] T 21ee3d7c6f764d0483b335da40e61efc P 7796ea3fba294d17a34386873969431d [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:31.940609 29315 raft_consensus.cc:2932] T 21ee3d7c6f764d0483b335da40e61efc P 09258ac095664fa99caf0707262cde13 [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:31.943326 29037 catalog_manager.cc:5484] T 21ee3d7c6f764d0483b335da40e61efc P 09258ac095664fa99caf0707262cde13 reported cstate change: config changed from index 4 to 6, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) evicted. New cstate: current_term: 2 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:31.945895 29425 tablet_service.cc:1514] Processing DeleteTablet for tablet 21ee3d7c6f764d0483b335da40e61efc with delete_type TABLET_DATA_TOMBSTONED (TS dbcdc6ac635b44b185c3eba63f66e9d6 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:31.946106 6415 tablet_replica.cc:330] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:31.946255 6415 raft_consensus.cc:2229] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:31.946354 6415 raft_consensus.cc:2258] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:31.946861 6415 ts_tablet_manager.cc:1875] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:31.948726 6415 ts_tablet_manager.cc:1888] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 20:46:31.948773 6415 log.cc:1196] T 21ee3d7c6f764d0483b335da40e61efc P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/21ee3d7c6f764d0483b335da40e61efc
I20240502 20:46:31.949208 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet 21ee3d7c6f764d0483b335da40e61efc (table rebalance_test_table_13 [id=08ce6b9eef5e4b2da494180f1ee1852b]) successfully deleted
I20240502 20:46:31.970702 29175 consensus_queue.cc:237] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:31.972028 29315 raft_consensus.cc:1261] T 7e0f811a7bd24ab395364e9461144558 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:31.972077 31756 raft_consensus.cc:1261] T 7e0f811a7bd24ab395364e9461144558 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 20:46:31.972328 29119 consensus_peers.cc:485] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d -> Peer dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): Couldn't send request to peer dbcdc6ac635b44b185c3eba63f66e9d6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7e0f811a7bd24ab395364e9461144558. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:31.972443 6633 consensus_queue.cc:1035] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:31.972635 6409 consensus_queue.cc:1035] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:31.974406 6633 raft_consensus.cc:2932] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:31.974776 29315 raft_consensus.cc:2932] T 7e0f811a7bd24ab395364e9461144558 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:31.977790 31755 raft_consensus.cc:2932] T 7e0f811a7bd24ab395364e9461144558 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:31.978193 29041 catalog_manager.cc:5484] T 7e0f811a7bd24ab395364e9461144558 P 09258ac095664fa99caf0707262cde13 reported cstate change: config changed from index -1 to 2, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) added. New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:31.998208 29042 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:31.998518 6069 raft_consensus.cc:482] T 9d3d4caa84c74e4a960f911c1557f8ce P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:31.998615 6069 raft_consensus.cc:504] T 9d3d4caa84c74e4a960f911c1557f8ce P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:31.998972 6069 leader_election.cc:288] T 9d3d4caa84c74e4a960f911c1557f8ce P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:31.999229 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3d4caa84c74e4a960f911c1557f8ce" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:31.999354 31755 raft_consensus.cc:2454] T 9d3d4caa84c74e4a960f911c1557f8ce P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:31.999632 29510 leader_election.cc:302] T 9d3d4caa84c74e4a960f911c1557f8ce P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:31.999790 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3d4caa84c74e4a960f911c1557f8ce" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:31.999893 29175 raft_consensus.cc:2454] T 9d3d4caa84c74e4a960f911c1557f8ce P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:32.000053 6069 raft_consensus.cc:2781] T 9d3d4caa84c74e4a960f911c1557f8ce P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:32.000109 6069 raft_consensus.cc:482] T 9d3d4caa84c74e4a960f911c1557f8ce P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:32.000144 6069 raft_consensus.cc:3037] T 9d3d4caa84c74e4a960f911c1557f8ce P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:32.000967 6069 raft_consensus.cc:504] T 9d3d4caa84c74e4a960f911c1557f8ce P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:32.001232 6069 leader_election.cc:288] T 9d3d4caa84c74e4a960f911c1557f8ce P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Requested vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:32.001533 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3d4caa84c74e4a960f911c1557f8ce" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:32.001629 31755 raft_consensus.cc:3037] T 9d3d4caa84c74e4a960f911c1557f8ce P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:32.001688 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3d4caa84c74e4a960f911c1557f8ce" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:32.001791 29175 raft_consensus.cc:3037] T 9d3d4caa84c74e4a960f911c1557f8ce P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:32.002439 31755 raft_consensus.cc:2454] T 9d3d4caa84c74e4a960f911c1557f8ce P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:32.002588 29175 raft_consensus.cc:2454] T 9d3d4caa84c74e4a960f911c1557f8ce P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:32.002717 29510 leader_election.cc:302] T 9d3d4caa84c74e4a960f911c1557f8ce P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:32.002873 6069 raft_consensus.cc:2781] T 9d3d4caa84c74e4a960f911c1557f8ce P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:32.002959 6069 raft_consensus.cc:686] T 9d3d4caa84c74e4a960f911c1557f8ce P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Becoming Leader. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Running, Role: LEADER
I20240502 20:46:32.003095 6069 consensus_queue.cc:237] T 9d3d4caa84c74e4a960f911c1557f8ce P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:32.005898 29042 catalog_manager.cc:5484] T 9d3d4caa84c74e4a960f911c1557f8ce P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: term changed from 0 to 1, leader changed from <none> to db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:32.006991 6593 tablet.cc:1722] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:32.007282 6593 tablet.cc:1620] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:32.011451 6113 tablet.cc:1722] T 8c95bdf553804cd6bab1ecdcf2158faf P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:32.011731 6113 tablet.cc:1620] T 8c95bdf553804cd6bab1ecdcf2158faf P 09258ac095664fa99caf0707262cde13: MemRowSet was empty: no flush needed.
I20240502 20:46:32.015611 6371 tablet.cc:1722] T 8c95bdf553804cd6bab1ecdcf2158faf P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:32.015877 6371 tablet.cc:1620] T 8c95bdf553804cd6bab1ecdcf2158faf P dbcdc6ac635b44b185c3eba63f66e9d6: MemRowSet was empty: no flush needed.
I20240502 20:46:32.020712 29037 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=52f922a238a54a2c8224d3fb855ed6bf] alter complete (version 2)
I20240502 20:46:32.027412 6583 raft_consensus.cc:482] T 6e80723ee7624d698e9e0a0bd494e1f0 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:32.027514 6583 raft_consensus.cc:504] T 6e80723ee7624d698e9e0a0bd494e1f0 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:32.027990 6583 leader_election.cc:288] T 6e80723ee7624d698e9e0a0bd494e1f0 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:32.028220 31886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e80723ee7624d698e9e0a0bd494e1f0" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:32.028334 31886 raft_consensus.cc:2454] T 6e80723ee7624d698e9e0a0bd494e1f0 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:32.028340 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e80723ee7624d698e9e0a0bd494e1f0" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:32.028458 31495 raft_consensus.cc:2454] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:32.028700 31560 leader_election.cc:302] T 6e80723ee7624d698e9e0a0bd494e1f0 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:32.028930 6583 raft_consensus.cc:2781] T 6e80723ee7624d698e9e0a0bd494e1f0 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:32.028975 6583 raft_consensus.cc:482] T 6e80723ee7624d698e9e0a0bd494e1f0 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:32.029004 6583 raft_consensus.cc:3037] T 6e80723ee7624d698e9e0a0bd494e1f0 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:32.029982 6583 raft_consensus.cc:504] T 6e80723ee7624d698e9e0a0bd494e1f0 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:32.030299 6583 leader_election.cc:288] T 6e80723ee7624d698e9e0a0bd494e1f0 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Requested vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:32.030740 31886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e80723ee7624d698e9e0a0bd494e1f0" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:32.030849 31886 raft_consensus.cc:3037] T 6e80723ee7624d698e9e0a0bd494e1f0 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:32.031348 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e80723ee7624d698e9e0a0bd494e1f0" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:32.031457 31495 raft_consensus.cc:3037] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:32.031759 31886 raft_consensus.cc:2454] T 6e80723ee7624d698e9e0a0bd494e1f0 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:32.032042 31560 leader_election.cc:302] T 6e80723ee7624d698e9e0a0bd494e1f0 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:32.032212 6583 raft_consensus.cc:2781] T 6e80723ee7624d698e9e0a0bd494e1f0 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:32.032287 6583 raft_consensus.cc:686] T 6e80723ee7624d698e9e0a0bd494e1f0 P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Becoming Leader. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Running, Role: LEADER
I20240502 20:46:32.032354 31495 raft_consensus.cc:2454] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:32.032438 6583 consensus_queue.cc:237] T 6e80723ee7624d698e9e0a0bd494e1f0 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:32.034677 29042 catalog_manager.cc:5484] T 6e80723ee7624d698e9e0a0bd494e1f0 P 315915efcf2b413ca7f9d874338e1b2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70). New cstate: current_term: 1 leader_uuid: "315915efcf2b413ca7f9d874338e1b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:32.123330 29035 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:32.128263 6593 tablet.cc:1722] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:32.128633 6113 tablet.cc:1722] T 8c95bdf553804cd6bab1ecdcf2158faf P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:32.130388 6371 tablet.cc:1722] T 8c95bdf553804cd6bab1ecdcf2158faf P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:32.134161 29035 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=52f922a238a54a2c8224d3fb855ed6bf] alter complete (version 3)
I20240502 20:46:32.235342 29035 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:32.235481 29035 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:32.237054 3289 raft_consensus.cc:482] T af74e9afe64347168b6e28eac41e9860 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:32.237157 3289 raft_consensus.cc:504] T af74e9afe64347168b6e28eac41e9860 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:32.237552 3289 leader_election.cc:288] T af74e9afe64347168b6e28eac41e9860 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:32.238286 29035 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 8c95bdf553804cd6bab1ecdcf2158faf
I20240502 20:46:32.238397 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af74e9afe64347168b6e28eac41e9860" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:32.238559 31495 raft_consensus.cc:2454] T af74e9afe64347168b6e28eac41e9860 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:32.238837 31692 leader_election.cc:302] T af74e9afe64347168b6e28eac41e9860 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:32.238899 31868 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c95bdf553804cd6bab1ecdcf2158faf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:32 UTC) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:32.238986 29425 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c95bdf553804cd6bab1ecdcf2158faf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:32 UTC) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:32.239012 3289 raft_consensus.cc:2781] T af74e9afe64347168b6e28eac41e9860 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:32.239058 3289 raft_consensus.cc:482] T af74e9afe64347168b6e28eac41e9860 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:32.239090 3289 raft_consensus.cc:3037] T af74e9afe64347168b6e28eac41e9860 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:32.239142 6415 tablet_replica.cc:330] T 8c95bdf553804cd6bab1ecdcf2158faf P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:32.239192 6679 tablet_replica.cc:330] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:32.239286 6415 raft_consensus.cc:2229] T 8c95bdf553804cd6bab1ecdcf2158faf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:32.239369 6679 raft_consensus.cc:2229] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:32.239391 6415 raft_consensus.cc:2258] T 8c95bdf553804cd6bab1ecdcf2158faf P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:32.239539 6679 raft_consensus.cc:2258] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:32.239542 29294 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c95bdf553804cd6bab1ecdcf2158faf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:32 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:32.239807 6680 tablet_replica.cc:330] T 8c95bdf553804cd6bab1ecdcf2158faf P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:32.239969 6680 raft_consensus.cc:2229] T 8c95bdf553804cd6bab1ecdcf2158faf P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:32.240083 6415 ts_tablet_manager.cc:1875] T 8c95bdf553804cd6bab1ecdcf2158faf P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:32.240134 6680 raft_consensus.cc:2258] T 8c95bdf553804cd6bab1ecdcf2158faf P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:32.240183 3289 raft_consensus.cc:504] T af74e9afe64347168b6e28eac41e9860 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:32.240520 3289 leader_election.cc:288] T af74e9afe64347168b6e28eac41e9860 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:32.240638 6680 ts_tablet_manager.cc:1875] T 8c95bdf553804cd6bab1ecdcf2158faf P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:32.240731 6679 ts_tablet_manager.cc:1875] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:32.240974 29313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af74e9afe64347168b6e28eac41e9860" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:32.241080 29313 raft_consensus.cc:3037] T af74e9afe64347168b6e28eac41e9860 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:32.241959 29313 raft_consensus.cc:2454] T af74e9afe64347168b6e28eac41e9860 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:32.242234 31692 leader_election.cc:302] T af74e9afe64347168b6e28eac41e9860 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:32.242245 6415 ts_tablet_manager.cc:1888] T 8c95bdf553804cd6bab1ecdcf2158faf P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:32.242311 6415 log.cc:1196] T 8c95bdf553804cd6bab1ecdcf2158faf P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/8c95bdf553804cd6bab1ecdcf2158faf
I20240502 20:46:32.242583 6415 ts_tablet_manager.cc:1909] T 8c95bdf553804cd6bab1ecdcf2158faf P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting consensus metadata
I20240502 20:46:32.242602 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af74e9afe64347168b6e28eac41e9860" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:32.242702 31495 raft_consensus.cc:3037] T af74e9afe64347168b6e28eac41e9860 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:32.243028 6679 ts_tablet_manager.cc:1888] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:32.243484 6679 log.cc:1196] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/8c95bdf553804cd6bab1ecdcf2158faf
I20240502 20:46:32.243603 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet 8c95bdf553804cd6bab1ecdcf2158faf (table rebalancer_dynamic_table_new_name [id=52f922a238a54a2c8224d3fb855ed6bf]) successfully deleted
I20240502 20:46:32.243680 31495 raft_consensus.cc:2454] T af74e9afe64347168b6e28eac41e9860 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:32.243685 6680 ts_tablet_manager.cc:1888] T 8c95bdf553804cd6bab1ecdcf2158faf P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:32.243770 6680 log.cc:1196] T 8c95bdf553804cd6bab1ecdcf2158faf P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/8c95bdf553804cd6bab1ecdcf2158faf
I20240502 20:46:32.243860 6679 ts_tablet_manager.cc:1909] T 8c95bdf553804cd6bab1ecdcf2158faf P e6b60803d0764d228c863e2da65e3691: Deleting consensus metadata
I20240502 20:46:32.244089 6680 ts_tablet_manager.cc:1909] T 8c95bdf553804cd6bab1ecdcf2158faf P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:32.244119 6681 raft_consensus.cc:2781] T af74e9afe64347168b6e28eac41e9860 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:32.244199 6681 raft_consensus.cc:686] T af74e9afe64347168b6e28eac41e9860 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Becoming Leader. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Running, Role: LEADER
I20240502 20:46:32.244446 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af74e9afe64347168b6e28eac41e9860" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:32.244491 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet 8c95bdf553804cd6bab1ecdcf2158faf (table rebalancer_dynamic_table_new_name [id=52f922a238a54a2c8224d3fb855ed6bf]) successfully deleted
I20240502 20:46:32.244328 6681 consensus_queue.cc:237] T af74e9afe64347168b6e28eac41e9860 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:32.244570 29315 raft_consensus.cc:2362] T af74e9afe64347168b6e28eac41e9860 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1. Re-sending same reply.
I20240502 20:46:32.244815 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet 8c95bdf553804cd6bab1ecdcf2158faf (table rebalancer_dynamic_table_new_name [id=52f922a238a54a2c8224d3fb855ed6bf]) successfully deleted
I20240502 20:46:32.246652 29035 catalog_manager.cc:5484] T af74e9afe64347168b6e28eac41e9860 P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: term changed from 0 to 1, leader changed from <none> to e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71). New cstate: current_term: 1 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:32.276674 31475 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:32.278708 31602 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:32.279610 29294 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:32.281211 31868 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:32.281208 31736 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:32.282775 29162 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 20:46:32.289453 29425 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 20:46:32.301537 29557 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:32.344256 29035 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:32.350808 29425 tablet_service.cc:1467] Processing CreateTablet for tablet 9c89a184b7df434a82779586187e43d6 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=5ae5f3d7efbf49b0a78c9d9e0e37cfae]), partition=
I20240502 20:46:32.350977 29162 tablet_service.cc:1467] Processing CreateTablet for tablet 9c89a184b7df434a82779586187e43d6 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=5ae5f3d7efbf49b0a78c9d9e0e37cfae]), partition=
I20240502 20:46:32.351040 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c89a184b7df434a82779586187e43d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:32.351209 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c89a184b7df434a82779586187e43d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:32.352139 31868 tablet_service.cc:1467] Processing CreateTablet for tablet 9c89a184b7df434a82779586187e43d6 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=5ae5f3d7efbf49b0a78c9d9e0e37cfae]), partition=
I20240502 20:46:32.352362 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c89a184b7df434a82779586187e43d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:32.353083 6698 tablet_bootstrap.cc:492] T 9c89a184b7df434a82779586187e43d6 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:32.353953 6698 tablet_bootstrap.cc:654] T 9c89a184b7df434a82779586187e43d6 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:32.354323 6578 tablet_bootstrap.cc:492] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:32.355177 6578 tablet_bootstrap.cc:654] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:32.355816 6537 tablet_bootstrap.cc:492] T 9c89a184b7df434a82779586187e43d6 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:32.356827 6537 tablet_bootstrap.cc:654] T 9c89a184b7df434a82779586187e43d6 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:32.357800 6537 tablet_bootstrap.cc:492] T 9c89a184b7df434a82779586187e43d6 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:32.357884 6537 ts_tablet_manager.cc:1367] T 9c89a184b7df434a82779586187e43d6 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:32.358274 6537 raft_consensus.cc:348] T 9c89a184b7df434a82779586187e43d6 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:32.358369 6537 raft_consensus.cc:374] T 9c89a184b7df434a82779586187e43d6 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:32.358394 6537 raft_consensus.cc:729] T 9c89a184b7df434a82779586187e43d6 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:32.358527 6537 consensus_queue.cc:260] T 9c89a184b7df434a82779586187e43d6 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:32.358804 6537 ts_tablet_manager.cc:1398] T 9c89a184b7df434a82779586187e43d6 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:32.363719 6578 tablet_bootstrap.cc:492] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:32.363814 6578 ts_tablet_manager.cc:1367] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240502 20:46:32.364239 6578 raft_consensus.cc:348] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:32.364329 6578 raft_consensus.cc:374] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:32.364358 6578 raft_consensus.cc:729] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:32.364511 6578 consensus_queue.cc:260] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:32.364768 6578 ts_tablet_manager.cc:1398] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:32.365687 6698 tablet_bootstrap.cc:492] T 9c89a184b7df434a82779586187e43d6 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:32.365794 6698 ts_tablet_manager.cc:1367] T 9c89a184b7df434a82779586187e43d6 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 20:46:32.366202 6698 raft_consensus.cc:348] T 9c89a184b7df434a82779586187e43d6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:32.366302 6698 raft_consensus.cc:374] T 9c89a184b7df434a82779586187e43d6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:32.366353 6698 raft_consensus.cc:729] T 9c89a184b7df434a82779586187e43d6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:32.366477 6698 consensus_queue.cc:260] T 9c89a184b7df434a82779586187e43d6 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:32.366755 6698 ts_tablet_manager.cc:1398] T 9c89a184b7df434a82779586187e43d6 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:32.395787 6069 consensus_queue.cc:1035] T 9d3d4caa84c74e4a960f911c1557f8ce P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:32.398648 5346 consensus_queue.cc:1035] T 9d3d4caa84c74e4a960f911c1557f8ce P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:32.453087 5804 consensus_queue.cc:1035] T 6e80723ee7624d698e9e0a0bd494e1f0 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:32.460079 6583 consensus_queue.cc:1035] T 6e80723ee7624d698e9e0a0bd494e1f0 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:32.469566 6633 raft_consensus.cc:482] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:32.469656 6633 raft_consensus.cc:504] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:32.470036 6633 leader_election.cc:288] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:32.470260 31886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c89a184b7df434a82779586187e43d6" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:32.470379 31886 raft_consensus.cc:2454] T 9c89a184b7df434a82779586187e43d6 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:32.470575 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c89a184b7df434a82779586187e43d6" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:32.470698 29444 raft_consensus.cc:2454] T 9c89a184b7df434a82779586187e43d6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:32.470690 29118 leader_election.cc:302] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:32.471001 6633 raft_consensus.cc:2781] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:32.471062 6633 raft_consensus.cc:482] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:32.471099 6633 raft_consensus.cc:3037] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:32.472112 6633 raft_consensus.cc:504] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:32.472414 6633 leader_election.cc:288] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:32.472674 31886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c89a184b7df434a82779586187e43d6" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:32.472775 31886 raft_consensus.cc:3037] T 9c89a184b7df434a82779586187e43d6 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:32.472910 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c89a184b7df434a82779586187e43d6" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:32.473006 29444 raft_consensus.cc:3037] T 9c89a184b7df434a82779586187e43d6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:32.473809 31886 raft_consensus.cc:2454] T 9c89a184b7df434a82779586187e43d6 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:32.473807 29444 raft_consensus.cc:2454] T 9c89a184b7df434a82779586187e43d6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:32.474072 29118 leader_election.cc:302] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:32.474253 6633 raft_consensus.cc:2781] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:32.474356 6633 raft_consensus.cc:686] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:32.474509 6633 consensus_queue.cc:237] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:32.476229 29036 catalog_manager.cc:5484] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:32.493696 31475 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:32.494758 29294 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:32.505254 29162 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 20:46:32.507346 31602 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:32.511814 31736 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:32.525152 29557 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:32.525743 31868 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:32.540794 6761 ts_tablet_manager.cc:902] T 7e0f811a7bd24ab395364e9461144558 P dbcdc6ac635b44b185c3eba63f66e9d6: Initiating tablet copy from peer 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:32.541333 6761 tablet_copy_client.cc:308] T 7e0f811a7bd24ab395364e9461144558 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.65:43267
I20240502 20:46:32.541765 29202 tablet_copy_service.cc:140] P 7796ea3fba294d17a34386873969431d: Received BeginTabletCopySession request for tablet 7e0f811a7bd24ab395364e9461144558 from peer dbcdc6ac635b44b185c3eba63f66e9d6 ({username='slave'} at 127.22.45.67:45133)
I20240502 20:46:32.541862 29202 tablet_copy_service.cc:161] P 7796ea3fba294d17a34386873969431d: Beginning new tablet copy session on tablet 7e0f811a7bd24ab395364e9461144558 from peer dbcdc6ac635b44b185c3eba63f66e9d6 at {username='slave'} at 127.22.45.67:45133: session id = dbcdc6ac635b44b185c3eba63f66e9d6-7e0f811a7bd24ab395364e9461144558
I20240502 20:46:32.542783 29202 tablet_copy_source_session.cc:205] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:32.543741 6761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e0f811a7bd24ab395364e9461144558. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:32.556221 29425 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 20:46:32.557639 6761 tablet_copy_client.cc:783] T 7e0f811a7bd24ab395364e9461144558 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:32.557857 6761 tablet_copy_client.cc:647] T 7e0f811a7bd24ab395364e9461144558 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:32.558854 6761 tablet_copy_client.cc:523] T 7e0f811a7bd24ab395364e9461144558 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:32.560582 6761 tablet_bootstrap.cc:492] T 7e0f811a7bd24ab395364e9461144558 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:32.568092 6761 tablet_bootstrap.cc:492] T 7e0f811a7bd24ab395364e9461144558 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:32.568394 6761 tablet_bootstrap.cc:492] T 7e0f811a7bd24ab395364e9461144558 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap complete.
I20240502 20:46:32.568619 6761 ts_tablet_manager.cc:1367] T 7e0f811a7bd24ab395364e9461144558 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20240502 20:46:32.569092 6761 raft_consensus.cc:348] T 7e0f811a7bd24ab395364e9461144558 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:32.569275 6761 raft_consensus.cc:729] T 7e0f811a7bd24ab395364e9461144558 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:32.569690 6761 consensus_queue.cc:260] T 7e0f811a7bd24ab395364e9461144558 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:32.570066 6761 ts_tablet_manager.cc:1398] T 7e0f811a7bd24ab395364e9461144558 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:32.570683 29202 tablet_copy_service.cc:342] P 7796ea3fba294d17a34386873969431d: Request end of tablet copy session dbcdc6ac635b44b185c3eba63f66e9d6-7e0f811a7bd24ab395364e9461144558 received from {username='slave'} at 127.22.45.67:45133
I20240502 20:46:32.570767 29202 tablet_copy_service.cc:433] P 7796ea3fba294d17a34386873969431d: ending tablet copy session dbcdc6ac635b44b185c3eba63f66e9d6-7e0f811a7bd24ab395364e9461144558 on tablet 7e0f811a7bd24ab395364e9461144558 with peer dbcdc6ac635b44b185c3eba63f66e9d6
I20240502 20:46:32.588336 29036 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:32.593050 29444 raft_consensus.cc:1261] T 9c89a184b7df434a82779586187e43d6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:32.593456 6633 consensus_queue.cc:1035] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:32.593855 31886 raft_consensus.cc:1261] T 9c89a184b7df434a82779586187e43d6 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:32.594202 6633 consensus_queue.cc:1035] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:32.597056 29036 catalog_manager.cc:5576] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095) does not have the latest schema for tablet 9c89a184b7df434a82779586187e43d6 (table rebalancer_dynamic_table [id=5ae5f3d7efbf49b0a78c9d9e0e37cfae]). Expected version 1 got 0
I20240502 20:46:32.596527 6780 tablet.cc:1722] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:32.597554 6780 tablet.cc:1620] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d: MemRowSet was empty: no flush needed.
I20240502 20:46:32.597851 29035 catalog_manager.cc:5576] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207) does not have the latest schema for tablet 9c89a184b7df434a82779586187e43d6 (table rebalancer_dynamic_table [id=5ae5f3d7efbf49b0a78c9d9e0e37cfae]). Expected version 1 got 0
I20240502 20:46:32.598299 6593 tablet.cc:1722] T 9c89a184b7df434a82779586187e43d6 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:32.598608 6593 tablet.cc:1620] T 9c89a184b7df434a82779586187e43d6 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:32.600724 6284 mvcc.cc:205] Tried to move back new op lower bound from 7023340718458109952 to 7023340717976748032. Current Snapshot: MvccSnapshot[applied={T|T < 7023340718458109952}]
I20240502 20:46:32.600906 6371 tablet.cc:1722] T 9c89a184b7df434a82779586187e43d6 P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:32.601174 6371 tablet.cc:1620] T 9c89a184b7df434a82779586187e43d6 P dbcdc6ac635b44b185c3eba63f66e9d6: MemRowSet was empty: no flush needed.
I20240502 20:46:32.604872 29036 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=5ae5f3d7efbf49b0a78c9d9e0e37cfae] alter complete (version 1)
I20240502 20:46:32.609349 29444 consensus_queue.cc:237] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:32.610504 29574 raft_consensus.cc:1261] T db9c5ce233da4b1a9ce70ae04aef7658 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:32.610522 31624 raft_consensus.cc:1261] T db9c5ce233da4b1a9ce70ae04aef7658 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 20:46:32.611068 6234 consensus_queue.cc:1035] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:32.611250 6237 consensus_queue.cc:1035] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:32.613903 6237 raft_consensus.cc:2932] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:32.614292 31624 raft_consensus.cc:2932] T db9c5ce233da4b1a9ce70ae04aef7658 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:32.616276 29574 raft_consensus.cc:2932] T db9c5ce233da4b1a9ce70ae04aef7658 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:32.617151 29036 catalog_manager.cc:5484] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index 4 to 5, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) evicted. New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:32.620352 29294 tablet_service.cc:1514] Processing DeleteTablet for tablet db9c5ce233da4b1a9ce70ae04aef7658 with delete_type TABLET_DATA_TOMBSTONED (TS 09258ac095664fa99caf0707262cde13 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:32.621062 6680 tablet_replica.cc:330] T db9c5ce233da4b1a9ce70ae04aef7658 P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:32.621214 6680 raft_consensus.cc:2229] T db9c5ce233da4b1a9ce70ae04aef7658 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:32.621352 6680 raft_consensus.cc:2258] T db9c5ce233da4b1a9ce70ae04aef7658 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:32.621902 6680 ts_tablet_manager.cc:1875] T db9c5ce233da4b1a9ce70ae04aef7658 P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:32.623878 6680 ts_tablet_manager.cc:1888] T db9c5ce233da4b1a9ce70ae04aef7658 P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 20:46:32.623960 6680 log.cc:1196] T db9c5ce233da4b1a9ce70ae04aef7658 P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/db9c5ce233da4b1a9ce70ae04aef7658
I20240502 20:46:32.624411 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet db9c5ce233da4b1a9ce70ae04aef7658 (table rebalance_test_table_1 [id=8011864bab7b4506a89a8bbb019d244f]) successfully deleted
I20240502 20:46:32.665149 3289 consensus_queue.cc:1035] T af74e9afe64347168b6e28eac41e9860 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:32.667776 6681 consensus_queue.cc:1035] T af74e9afe64347168b6e28eac41e9860 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:32.709380 29042 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:32.717599 6780 tablet.cc:1722] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:32.717916 6780 tablet.cc:1620] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d: MemRowSet was empty: no flush needed.
I20240502 20:46:32.718276 6593 tablet.cc:1722] T 9c89a184b7df434a82779586187e43d6 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:32.718588 6593 tablet.cc:1620] T 9c89a184b7df434a82779586187e43d6 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:32.720561 6371 tablet.cc:1722] T 9c89a184b7df434a82779586187e43d6 P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:32.720851 6371 tablet.cc:1620] T 9c89a184b7df434a82779586187e43d6 P dbcdc6ac635b44b185c3eba63f66e9d6: MemRowSet was empty: no flush needed.
I20240502 20:46:32.731005 29040 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=5ae5f3d7efbf49b0a78c9d9e0e37cfae] alter complete (version 2)
I20240502 20:46:32.754029 31602 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:32.758981 29294 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:32.763937 31868 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:32.765532 29425 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 20:46:32.771514 29162 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 20:46:32.780990 31475 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:32.783318 29557 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:32.811888 31736 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 20:46:32.816231 29037 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:32.822782 29425 tablet_service.cc:1467] Processing CreateTablet for tablet 3126707d1c8c4c1786da5bfdfce53eb0 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=73f9b40350d24e0c95e468bd0878b4d5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:32.822957 29162 tablet_service.cc:1467] Processing CreateTablet for tablet 3126707d1c8c4c1786da5bfdfce53eb0 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=73f9b40350d24e0c95e468bd0878b4d5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:32.823071 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3126707d1c8c4c1786da5bfdfce53eb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:32.823187 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3126707d1c8c4c1786da5bfdfce53eb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:32.823784 29294 tablet_service.cc:1467] Processing CreateTablet for tablet 73c2e0de3fcf455ba1fecf701d23a05f (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=73f9b40350d24e0c95e468bd0878b4d5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:32.824024 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73c2e0de3fcf455ba1fecf701d23a05f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:32.825418 29556 tablet_service.cc:1467] Processing CreateTablet for tablet c8dc196b2d4e48b195524bc79c07b9eb (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=73f9b40350d24e0c95e468bd0878b4d5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:32.825529 6578 tablet_bootstrap.cc:492] T 3126707d1c8c4c1786da5bfdfce53eb0 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:32.825697 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8dc196b2d4e48b195524bc79c07b9eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:32.826442 29295 tablet_service.cc:1467] Processing CreateTablet for tablet c8dc196b2d4e48b195524bc79c07b9eb (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=73f9b40350d24e0c95e468bd0878b4d5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:32.826668 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8dc196b2d4e48b195524bc79c07b9eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:32.826956 6863 tablet_bootstrap.cc:492] T 73c2e0de3fcf455ba1fecf701d23a05f P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:32.827153 6578 tablet_bootstrap.cc:654] T 3126707d1c8c4c1786da5bfdfce53eb0 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:32.827883 6698 tablet_bootstrap.cc:492] T 3126707d1c8c4c1786da5bfdfce53eb0 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:32.827754 6863 tablet_bootstrap.cc:654] T 73c2e0de3fcf455ba1fecf701d23a05f P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:32.828255 6578 tablet_bootstrap.cc:492] T 3126707d1c8c4c1786da5bfdfce53eb0 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:32.828336 6578 ts_tablet_manager.cc:1367] T 3126707d1c8c4c1786da5bfdfce53eb0 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 20:46:32.828734 6698 tablet_bootstrap.cc:654] T 3126707d1c8c4c1786da5bfdfce53eb0 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:32.828763 6578 raft_consensus.cc:348] T 3126707d1c8c4c1786da5bfdfce53eb0 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:32.828860 6578 raft_consensus.cc:374] T 3126707d1c8c4c1786da5bfdfce53eb0 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:32.828895 6578 raft_consensus.cc:729] T 3126707d1c8c4c1786da5bfdfce53eb0 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:32.829020 6578 consensus_queue.cc:260] T 3126707d1c8c4c1786da5bfdfce53eb0 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:32.829288 6578 ts_tablet_manager.cc:1398] T 3126707d1c8c4c1786da5bfdfce53eb0 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:32.829373 31736 tablet_service.cc:1467] Processing CreateTablet for tablet 73c2e0de3fcf455ba1fecf701d23a05f (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=73f9b40350d24e0c95e468bd0878b4d5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:32.829613 31736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73c2e0de3fcf455ba1fecf701d23a05f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:32.825418 29557 tablet_service.cc:1467] Processing CreateTablet for tablet 3126707d1c8c4c1786da5bfdfce53eb0 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=73f9b40350d24e0c95e468bd0878b4d5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:32.830246 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3126707d1c8c4c1786da5bfdfce53eb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:32.830600 29162 tablet_service.cc:1467] Processing CreateTablet for tablet c8dc196b2d4e48b195524bc79c07b9eb (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=73f9b40350d24e0c95e468bd0878b4d5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:32.830798 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8dc196b2d4e48b195524bc79c07b9eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:32.832607 29040 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:32.832630 29425 tablet_service.cc:1467] Processing CreateTablet for tablet 73c2e0de3fcf455ba1fecf701d23a05f (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=73f9b40350d24e0c95e468bd0878b4d5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:32.832850 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73c2e0de3fcf455ba1fecf701d23a05f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:32.835041 6863 tablet_bootstrap.cc:492] T 73c2e0de3fcf455ba1fecf701d23a05f P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:32.835139 6863 ts_tablet_manager.cc:1367] T 73c2e0de3fcf455ba1fecf701d23a05f P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 20:46:32.835577 6863 raft_consensus.cc:348] T 73c2e0de3fcf455ba1fecf701d23a05f P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:32.835678 6863 raft_consensus.cc:374] T 73c2e0de3fcf455ba1fecf701d23a05f P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:32.835709 6863 raft_consensus.cc:729] T 73c2e0de3fcf455ba1fecf701d23a05f P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:32.835808 6863 consensus_queue.cc:260] T 73c2e0de3fcf455ba1fecf701d23a05f P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:32.835987 6865 tablet_bootstrap.cc:492] T c8dc196b2d4e48b195524bc79c07b9eb P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:32.836066 6863 ts_tablet_manager.cc:1398] T 73c2e0de3fcf455ba1fecf701d23a05f P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:32.836213 6863 tablet_bootstrap.cc:492] T c8dc196b2d4e48b195524bc79c07b9eb P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:32.837081 6865 tablet_bootstrap.cc:654] T c8dc196b2d4e48b195524bc79c07b9eb P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:32.837069 6863 tablet_bootstrap.cc:654] T c8dc196b2d4e48b195524bc79c07b9eb P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:32.837745 6578 tablet_bootstrap.cc:492] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:32.837920 6698 tablet_bootstrap.cc:492] T 3126707d1c8c4c1786da5bfdfce53eb0 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:32.837992 6698 ts_tablet_manager.cc:1367] T 3126707d1c8c4c1786da5bfdfce53eb0 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240502 20:46:32.838415 6698 raft_consensus.cc:348] T 3126707d1c8c4c1786da5bfdfce53eb0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:32.838546 6698 raft_consensus.cc:374] T 3126707d1c8c4c1786da5bfdfce53eb0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:32.838580 6698 raft_consensus.cc:729] T 3126707d1c8c4c1786da5bfdfce53eb0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:32.838647 6578 tablet_bootstrap.cc:654] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:32.838693 6698 consensus_queue.cc:260] T 3126707d1c8c4c1786da5bfdfce53eb0 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:32.839000 6698 ts_tablet_manager.cc:1398] T 3126707d1c8c4c1786da5bfdfce53eb0 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:32.839159 6698 tablet_bootstrap.cc:492] T 73c2e0de3fcf455ba1fecf701d23a05f P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:32.839607 6868 tablet_bootstrap.cc:492] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:32.839646 6578 tablet_bootstrap.cc:492] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:32.839707 6578 ts_tablet_manager.cc:1367] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:32.839989 6698 tablet_bootstrap.cc:654] T 73c2e0de3fcf455ba1fecf701d23a05f P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:32.840081 6578 raft_consensus.cc:348] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:32.840163 6578 raft_consensus.cc:374] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:32.840188 6578 raft_consensus.cc:729] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:32.840292 6578 consensus_queue.cc:260] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:32.840525 6578 ts_tablet_manager.cc:1398] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:32.841331 6868 tablet_bootstrap.cc:654] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:32.843709 6865 tablet_bootstrap.cc:492] T c8dc196b2d4e48b195524bc79c07b9eb P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:32.843835 6865 ts_tablet_manager.cc:1367] T c8dc196b2d4e48b195524bc79c07b9eb P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 20:46:32.844256 6865 raft_consensus.cc:348] T c8dc196b2d4e48b195524bc79c07b9eb P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:32.844400 6865 raft_consensus.cc:374] T c8dc196b2d4e48b195524bc79c07b9eb P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:32.844476 6865 raft_consensus.cc:729] T c8dc196b2d4e48b195524bc79c07b9eb P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:32.844682 6865 consensus_queue.cc:260] T c8dc196b2d4e48b195524bc79c07b9eb P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:32.845026 6865 ts_tablet_manager.cc:1398] T c8dc196b2d4e48b195524bc79c07b9eb P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:32.845522 6863 tablet_bootstrap.cc:492] T c8dc196b2d4e48b195524bc79c07b9eb P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:32.845654 6863 ts_tablet_manager.cc:1367] T c8dc196b2d4e48b195524bc79c07b9eb P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 20:46:32.845943 6865 tablet_bootstrap.cc:492] T 3126707d1c8c4c1786da5bfdfce53eb0 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:32.846177 6863 raft_consensus.cc:348] T c8dc196b2d4e48b195524bc79c07b9eb P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:32.847416 6863 raft_consensus.cc:374] T c8dc196b2d4e48b195524bc79c07b9eb P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:32.847502 6863 raft_consensus.cc:729] T c8dc196b2d4e48b195524bc79c07b9eb P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:32.848881 6780 tablet.cc:1722] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:32.847687 6863 consensus_queue.cc:260] T c8dc196b2d4e48b195524bc79c07b9eb P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:32.850400 6863 ts_tablet_manager.cc:1398] T c8dc196b2d4e48b195524bc79c07b9eb P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 20:46:32.851532 6371 tablet.cc:1722] T 9c89a184b7df434a82779586187e43d6 P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:32.854686 6868 tablet_bootstrap.cc:492] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:32.854818 6868 ts_tablet_manager.cc:1367] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 20:46:32.855283 6593 tablet.cc:1722] T 9c89a184b7df434a82779586187e43d6 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:32.855333 6868 raft_consensus.cc:348] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:32.855432 6868 raft_consensus.cc:374] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:32.855463 6868 raft_consensus.cc:729] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:32.855589 6868 consensus_queue.cc:260] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:32.855863 6868 ts_tablet_manager.cc:1398] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:32.859025 6865 tablet_bootstrap.cc:654] T 3126707d1c8c4c1786da5bfdfce53eb0 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:32.859751 29036 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=5ae5f3d7efbf49b0a78c9d9e0e37cfae] alter complete (version 3)
I20240502 20:46:32.863888 6865 tablet_bootstrap.cc:492] T 3126707d1c8c4c1786da5bfdfce53eb0 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:32.863963 6865 ts_tablet_manager.cc:1367] T 3126707d1c8c4c1786da5bfdfce53eb0 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240502 20:46:32.864370 6865 raft_consensus.cc:348] T 3126707d1c8c4c1786da5bfdfce53eb0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:32.864454 6865 raft_consensus.cc:374] T 3126707d1c8c4c1786da5bfdfce53eb0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:32.864480 6865 raft_consensus.cc:729] T 3126707d1c8c4c1786da5bfdfce53eb0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:32.864598 6865 consensus_queue.cc:260] T 3126707d1c8c4c1786da5bfdfce53eb0 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:32.864846 6865 ts_tablet_manager.cc:1398] T 3126707d1c8c4c1786da5bfdfce53eb0 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:32.872731 6698 tablet_bootstrap.cc:492] T 73c2e0de3fcf455ba1fecf701d23a05f P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:32.872818 6698 ts_tablet_manager.cc:1367] T 73c2e0de3fcf455ba1fecf701d23a05f P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.034s user 0.001s sys 0.000s
I20240502 20:46:32.873203 6698 raft_consensus.cc:348] T 73c2e0de3fcf455ba1fecf701d23a05f P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:32.873304 6698 raft_consensus.cc:374] T 73c2e0de3fcf455ba1fecf701d23a05f P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:32.873337 6698 raft_consensus.cc:729] T 73c2e0de3fcf455ba1fecf701d23a05f P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:32.873392 6777 raft_consensus.cc:482] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:32.873471 6777 raft_consensus.cc:504] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:32.873443 6698 consensus_queue.cc:260] T 73c2e0de3fcf455ba1fecf701d23a05f P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:32.873871 6777 leader_election.cc:288] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:32.873919 6698 ts_tablet_manager.cc:1398] T 73c2e0de3fcf455ba1fecf701d23a05f P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:32.874073 29313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8dc196b2d4e48b195524bc79c07b9eb" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:32.874193 29313 raft_consensus.cc:2454] T c8dc196b2d4e48b195524bc79c07b9eb P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:32.874737 29119 leader_election.cc:302] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:32.874933 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8dc196b2d4e48b195524bc79c07b9eb" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:32.874962 6777 raft_consensus.cc:2781] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:32.875011 6777 raft_consensus.cc:482] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:32.875042 6777 raft_consensus.cc:3037] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:32.875044 29574 raft_consensus.cc:2454] T c8dc196b2d4e48b195524bc79c07b9eb P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:32.875962 6777 raft_consensus.cc:504] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:32.876271 6777 leader_election.cc:288] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:32.876639 29313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8dc196b2d4e48b195524bc79c07b9eb" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:32.876735 29313 raft_consensus.cc:3037] T c8dc196b2d4e48b195524bc79c07b9eb P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:32.877604 29313 raft_consensus.cc:2454] T c8dc196b2d4e48b195524bc79c07b9eb P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:32.877624 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8dc196b2d4e48b195524bc79c07b9eb" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:32.877725 29574 raft_consensus.cc:3037] T c8dc196b2d4e48b195524bc79c07b9eb P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:32.877951 29119 leader_election.cc:302] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:32.878109 6777 raft_consensus.cc:2781] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:32.878178 6777 raft_consensus.cc:686] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:32.878288 6777 consensus_queue.cc:237] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:32.878767 29574 raft_consensus.cc:2454] T c8dc196b2d4e48b195524bc79c07b9eb P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:32.880647 29037 catalog_manager.cc:5484] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:32.962236 29042 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:32.962365 29042 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:32.964474 29042 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 9c89a184b7df434a82779586187e43d6
I20240502 20:46:32.964927 31868 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c89a184b7df434a82779586187e43d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:32 UTC) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:32.965013 29425 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c89a184b7df434a82779586187e43d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:32 UTC) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:32.965214 6922 tablet_replica.cc:330] T 9c89a184b7df434a82779586187e43d6 P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:32.965358 6922 raft_consensus.cc:2229] T 9c89a184b7df434a82779586187e43d6 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:32.965478 6922 raft_consensus.cc:2258] T 9c89a184b7df434a82779586187e43d6 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:32.966365 6922 ts_tablet_manager.cc:1875] T 9c89a184b7df434a82779586187e43d6 P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:32.967689 29162 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c89a184b7df434a82779586187e43d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:32 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:32.969164 6924 tablet_replica.cc:330] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:32.969456 6924 raft_consensus.cc:2229] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:32.969820 6924 raft_consensus.cc:2258] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:32.970698 6924 ts_tablet_manager.cc:1875] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:32.975975 6924 ts_tablet_manager.cc:1888] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:32.976190 6922 ts_tablet_manager.cc:1888] T 9c89a184b7df434a82779586187e43d6 P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:32.976280 6924 log.cc:1196] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/9c89a184b7df434a82779586187e43d6
I20240502 20:46:32.976343 6922 log.cc:1196] T 9c89a184b7df434a82779586187e43d6 P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/9c89a184b7df434a82779586187e43d6
I20240502 20:46:32.976728 6924 ts_tablet_manager.cc:1909] T 9c89a184b7df434a82779586187e43d6 P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:32.976768 6922 ts_tablet_manager.cc:1909] T 9c89a184b7df434a82779586187e43d6 P e6b60803d0764d228c863e2da65e3691: Deleting consensus metadata
I20240502 20:46:32.977420 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet 9c89a184b7df434a82779586187e43d6 (table rebalancer_dynamic_table_new_name [id=5ae5f3d7efbf49b0a78c9d9e0e37cfae]) successfully deleted
I20240502 20:46:32.977463 6923 tablet_replica.cc:330] T 9c89a184b7df434a82779586187e43d6 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:32.978878 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 9c89a184b7df434a82779586187e43d6 (table rebalancer_dynamic_table_new_name [id=5ae5f3d7efbf49b0a78c9d9e0e37cfae]) successfully deleted
I20240502 20:46:32.979106 6923 raft_consensus.cc:2229] T 9c89a184b7df434a82779586187e43d6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:32.979231 6923 raft_consensus.cc:2258] T 9c89a184b7df434a82779586187e43d6 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:32.980336 6923 ts_tablet_manager.cc:1875] T 9c89a184b7df434a82779586187e43d6 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:32.985072 6923 ts_tablet_manager.cc:1888] T 9c89a184b7df434a82779586187e43d6 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:32.985246 6923 log.cc:1196] T 9c89a184b7df434a82779586187e43d6 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/9c89a184b7df434a82779586187e43d6
I20240502 20:46:32.985651 6923 ts_tablet_manager.cc:1909] T 9c89a184b7df434a82779586187e43d6 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting consensus metadata
I20240502 20:46:32.986285 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet 9c89a184b7df434a82779586187e43d6 (table rebalancer_dynamic_table_new_name [id=5ae5f3d7efbf49b0a78c9d9e0e37cfae]) successfully deleted
I20240502 20:46:32.992453 29444 raft_consensus.cc:1203] T 7e0f811a7bd24ab395364e9461144558 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 20:46:32.993122 3289 raft_consensus.cc:482] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:32.993216 3289 raft_consensus.cc:504] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:32.993654 3289 leader_election.cc:288] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:32.994201 29313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73c2e0de3fcf455ba1fecf701d23a05f" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:32.994324 29313 raft_consensus.cc:2454] T 73c2e0de3fcf455ba1fecf701d23a05f P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:32.994626 31692 leader_election.cc:302] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:32.994814 6681 raft_consensus.cc:2781] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:32.994861 6681 raft_consensus.cc:482] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:32.994895 6681 raft_consensus.cc:3037] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:32.995370 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73c2e0de3fcf455ba1fecf701d23a05f" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:32.995481 29444 raft_consensus.cc:2454] T 73c2e0de3fcf455ba1fecf701d23a05f P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:32.995978 6681 raft_consensus.cc:504] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:32.996292 6681 leader_election.cc:288] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:32.996579 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73c2e0de3fcf455ba1fecf701d23a05f" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:32.996675 29444 raft_consensus.cc:3037] T 73c2e0de3fcf455ba1fecf701d23a05f P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:32.996735 29313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73c2e0de3fcf455ba1fecf701d23a05f" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:32.996846 29313 raft_consensus.cc:3037] T 73c2e0de3fcf455ba1fecf701d23a05f P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:32.997499 29444 raft_consensus.cc:2454] T 73c2e0de3fcf455ba1fecf701d23a05f P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:32.997565 29313 raft_consensus.cc:2454] T 73c2e0de3fcf455ba1fecf701d23a05f P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:32.997853 31692 leader_election.cc:302] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:32.998021 6681 raft_consensus.cc:2781] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:32.998102 6681 raft_consensus.cc:686] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Becoming Leader. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Running, Role: LEADER
I20240502 20:46:32.998224 6681 consensus_queue.cc:237] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:33.000312 29036 catalog_manager.cc:5484] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: term changed from 0 to 1, leader changed from <none> to e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71). New cstate: current_term: 1 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:33.066339 29040 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:33.071565 29425 tablet_service.cc:1467] Processing CreateTablet for tablet a2426b38ca0341339d254b225053f261 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=2374ce94fd564590bf63536fb6960027]), partition=
I20240502 20:46:33.071574 31475 tablet_service.cc:1467] Processing CreateTablet for tablet a2426b38ca0341339d254b225053f261 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=2374ce94fd564590bf63536fb6960027]), partition=
I20240502 20:46:33.071811 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2426b38ca0341339d254b225053f261. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:33.071811 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2426b38ca0341339d254b225053f261. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:33.073968 6698 tablet_bootstrap.cc:492] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:33.074070 6926 tablet_bootstrap.cc:492] T a2426b38ca0341339d254b225053f261 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:33.074342 31868 tablet_service.cc:1467] Processing CreateTablet for tablet a2426b38ca0341339d254b225053f261 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=2374ce94fd564590bf63536fb6960027]), partition=
I20240502 20:46:33.074594 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2426b38ca0341339d254b225053f261. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:33.074865 6698 tablet_bootstrap.cc:654] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:33.075842 6698 tablet_bootstrap.cc:492] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:33.075940 6698 ts_tablet_manager.cc:1367] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:33.076401 6698 raft_consensus.cc:348] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:33.076490 6698 raft_consensus.cc:374] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:33.076511 6698 raft_consensus.cc:729] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:33.076614 6698 consensus_queue.cc:260] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:33.076743 6928 tablet_bootstrap.cc:492] T a2426b38ca0341339d254b225053f261 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:33.076867 6698 ts_tablet_manager.cc:1398] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:33.077593 6928 tablet_bootstrap.cc:654] T a2426b38ca0341339d254b225053f261 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:33.077862 6926 tablet_bootstrap.cc:654] T a2426b38ca0341339d254b225053f261 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:33.078720 6928 tablet_bootstrap.cc:492] T a2426b38ca0341339d254b225053f261 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:33.078848 6928 ts_tablet_manager.cc:1367] T a2426b38ca0341339d254b225053f261 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:33.079226 6926 tablet_bootstrap.cc:492] T a2426b38ca0341339d254b225053f261 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:33.079293 6928 raft_consensus.cc:348] T a2426b38ca0341339d254b225053f261 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:33.079509 6928 raft_consensus.cc:374] T a2426b38ca0341339d254b225053f261 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:33.079582 6928 raft_consensus.cc:729] T a2426b38ca0341339d254b225053f261 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:33.079721 6928 consensus_queue.cc:260] T a2426b38ca0341339d254b225053f261 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:33.080803 6928 ts_tablet_manager.cc:1398] T a2426b38ca0341339d254b225053f261 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:33.081745 6926 ts_tablet_manager.cc:1367] T a2426b38ca0341339d254b225053f261 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 20:46:33.082137 6926 raft_consensus.cc:348] T a2426b38ca0341339d254b225053f261 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:33.082237 6926 raft_consensus.cc:374] T a2426b38ca0341339d254b225053f261 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:33.082304 6926 raft_consensus.cc:729] T a2426b38ca0341339d254b225053f261 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:33.082444 6926 consensus_queue.cc:260] T a2426b38ca0341339d254b225053f261 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:33.082798 6926 ts_tablet_manager.cc:1398] T a2426b38ca0341339d254b225053f261 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:33.185571 6789 raft_consensus.cc:482] T 3126707d1c8c4c1786da5bfdfce53eb0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:33.185699 6789 raft_consensus.cc:504] T 3126707d1c8c4c1786da5bfdfce53eb0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:33.186151 6789 leader_election.cc:288] T 3126707d1c8c4c1786da5bfdfce53eb0 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:33.186511 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3126707d1c8c4c1786da5bfdfce53eb0" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:33.186594 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3126707d1c8c4c1786da5bfdfce53eb0" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:33.186654 29182 raft_consensus.cc:2454] T 3126707d1c8c4c1786da5bfdfce53eb0 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:33.186730 29574 raft_consensus.cc:2454] T 3126707d1c8c4c1786da5bfdfce53eb0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:33.186906 29382 leader_election.cc:302] T 3126707d1c8c4c1786da5bfdfce53eb0 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:33.187055 6789 raft_consensus.cc:2781] T 3126707d1c8c4c1786da5bfdfce53eb0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:33.187115 6789 raft_consensus.cc:482] T 3126707d1c8c4c1786da5bfdfce53eb0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:33.187156 6789 raft_consensus.cc:3037] T 3126707d1c8c4c1786da5bfdfce53eb0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:33.187986 6789 raft_consensus.cc:504] T 3126707d1c8c4c1786da5bfdfce53eb0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:33.188247 6789 leader_election.cc:288] T 3126707d1c8c4c1786da5bfdfce53eb0 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:33.188510 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3126707d1c8c4c1786da5bfdfce53eb0" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:33.188614 29182 raft_consensus.cc:3037] T 3126707d1c8c4c1786da5bfdfce53eb0 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:33.188689 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3126707d1c8c4c1786da5bfdfce53eb0" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:33.188781 29574 raft_consensus.cc:3037] T 3126707d1c8c4c1786da5bfdfce53eb0 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:33.189565 29182 raft_consensus.cc:2454] T 3126707d1c8c4c1786da5bfdfce53eb0 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:33.189689 29574 raft_consensus.cc:2454] T 3126707d1c8c4c1786da5bfdfce53eb0 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:33.189855 29382 leader_election.cc:302] T 3126707d1c8c4c1786da5bfdfce53eb0 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:33.190135 6789 raft_consensus.cc:2781] T 3126707d1c8c4c1786da5bfdfce53eb0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:33.190218 6789 raft_consensus.cc:686] T 3126707d1c8c4c1786da5bfdfce53eb0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:33.190339 6789 consensus_queue.cc:237] T 3126707d1c8c4c1786da5bfdfce53eb0 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:33.192337 29036 catalog_manager.cc:5484] T 3126707d1c8c4c1786da5bfdfce53eb0 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:33.221694 6237 raft_consensus.cc:482] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:33.221796 6237 raft_consensus.cc:504] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:33.222186 6237 leader_election.cc:288] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:33.222486 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2426b38ca0341339d254b225053f261" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:33.222625 31495 raft_consensus.cc:2454] T a2426b38ca0341339d254b225053f261 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:33.222858 31886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2426b38ca0341339d254b225053f261" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:33.222980 31886 raft_consensus.cc:2454] T a2426b38ca0341339d254b225053f261 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:33.224821 29380 leader_election.cc:302] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dbcdc6ac635b44b185c3eba63f66e9d6, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:33.225131 6237 raft_consensus.cc:2781] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:33.225193 6237 raft_consensus.cc:482] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:33.225229 6237 raft_consensus.cc:3037] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:33.226176 6237 raft_consensus.cc:504] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:33.227183 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2426b38ca0341339d254b225053f261" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:33.227290 31495 raft_consensus.cc:3037] T a2426b38ca0341339d254b225053f261 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:33.228260 31495 raft_consensus.cc:2454] T a2426b38ca0341339d254b225053f261 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:33.228861 29295 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:33.233449 31475 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:33.233676 29381 leader_election.cc:302] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:33.233914 6789 raft_consensus.cc:2781] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:33.234009 6789 raft_consensus.cc:686] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:33.234135 6789 consensus_queue.cc:237] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:33.234964 29162 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 20:46:33.237008 29425 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 20:46:33.239171 29036 catalog_manager.cc:5484] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:33.240242 6237 leader_election.cc:288] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:33.240646 31886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2426b38ca0341339d254b225053f261" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:33.240751 31886 raft_consensus.cc:3037] T a2426b38ca0341339d254b225053f261 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:33.241688 31886 raft_consensus.cc:2454] T a2426b38ca0341339d254b225053f261 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:33.243404 29557 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:33.251987 31736 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:33.253137 31868 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:33.275171 6777 consensus_queue.cc:1035] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:33.283664 31602 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:33.287520 6247 consensus_queue.cc:1035] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:33.322846 29175 consensus_queue.cc:237] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:33.323865 31756 raft_consensus.cc:1261] T 7e0f811a7bd24ab395364e9461144558 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:33.323923 29444 raft_consensus.cc:1261] T 7e0f811a7bd24ab395364e9461144558 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:33.324290 6247 consensus_queue.cc:1035] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:33.324425 6633 consensus_queue.cc:1035] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:33.326021 6777 raft_consensus.cc:2932] T 7e0f811a7bd24ab395364e9461144558 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:33.326488 29444 raft_consensus.cc:2932] T 7e0f811a7bd24ab395364e9461144558 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:33.328022 31756 raft_consensus.cc:2932] T 7e0f811a7bd24ab395364e9461144558 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:33.330984 29036 catalog_manager.cc:5484] T 7e0f811a7bd24ab395364e9461144558 P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: config changed from index 2 to 3, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) evicted. New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:33.333410 29295 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e0f811a7bd24ab395364e9461144558 with delete_type TABLET_DATA_TOMBSTONED (TS 09258ac095664fa99caf0707262cde13 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:33.333694 6968 tablet_replica.cc:330] T 7e0f811a7bd24ab395364e9461144558 P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:33.333811 6968 raft_consensus.cc:2229] T 7e0f811a7bd24ab395364e9461144558 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:33.333909 6968 raft_consensus.cc:2258] T 7e0f811a7bd24ab395364e9461144558 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:33.334311 6968 ts_tablet_manager.cc:1875] T 7e0f811a7bd24ab395364e9461144558 P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:33.336061 6968 ts_tablet_manager.cc:1888] T 7e0f811a7bd24ab395364e9461144558 P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 20:46:33.336124 6968 log.cc:1196] T 7e0f811a7bd24ab395364e9461144558 P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/7e0f811a7bd24ab395364e9461144558
I20240502 20:46:33.336596 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet 7e0f811a7bd24ab395364e9461144558 (table rebalancer_extra_table_6 [id=9659763c7b3a4e2f8ffa64e3e9c5e095]) successfully deleted
I20240502 20:46:33.363951 29040 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:33.367682 31495 raft_consensus.cc:1261] T a2426b38ca0341339d254b225053f261 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:33.367697 31886 raft_consensus.cc:1261] T a2426b38ca0341339d254b225053f261 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:33.368068 6789 consensus_queue.cc:1035] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:33.368228 6237 consensus_queue.cc:1035] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:33.371639 29040 catalog_manager.cc:5576] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207) does not have the latest schema for tablet a2426b38ca0341339d254b225053f261 (table rebalancer_dynamic_table [id=2374ce94fd564590bf63536fb6960027]). Expected version 1 got 0
I20240502 20:46:33.373564 6974 tablet.cc:1722] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:33.373955 29042 catalog_manager.cc:5576] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887) does not have the latest schema for tablet a2426b38ca0341339d254b225053f261 (table rebalancer_dynamic_table [id=2374ce94fd564590bf63536fb6960027]). Expected version 1 got 0
I20240502 20:46:33.373968 6974 tablet.cc:1620] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6: MemRowSet was empty: no flush needed.
I20240502 20:46:33.376377 29042 catalog_manager.cc:5576] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887) does not have the latest schema for tablet a2426b38ca0341339d254b225053f261 (table rebalancer_dynamic_table [id=2374ce94fd564590bf63536fb6960027]). Expected version 1 got 0
I20240502 20:46:33.378679 29040 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=2374ce94fd564590bf63536fb6960027] alter complete (version 1)
I20240502 20:46:33.379312 29042 catalog_manager.cc:5576] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207) does not have the latest schema for tablet a2426b38ca0341339d254b225053f261 (table rebalancer_dynamic_table [id=2374ce94fd564590bf63536fb6960027]). Expected version 1 got 0
I20240502 20:46:33.380049 6976 tablet.cc:1722] T a2426b38ca0341339d254b225053f261 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:33.380331 6976 tablet.cc:1620] T a2426b38ca0341339d254b225053f261 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:33.380336 6975 tablet.cc:1722] T a2426b38ca0341339d254b225053f261 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:33.380661 6975 tablet.cc:1620] T a2426b38ca0341339d254b225053f261 P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:33.383540 3289 consensus_queue.cc:1035] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:33.386839 6681 consensus_queue.cc:1035] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:33.483410 29040 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:33.488473 6974 tablet.cc:1722] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:33.488813 6974 tablet.cc:1620] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6: MemRowSet was empty: no flush needed.
I20240502 20:46:33.489483 6975 tablet.cc:1722] T a2426b38ca0341339d254b225053f261 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:33.489799 6975 tablet.cc:1620] T a2426b38ca0341339d254b225053f261 P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:33.490554 6976 tablet.cc:1722] T a2426b38ca0341339d254b225053f261 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:33.490934 6976 tablet.cc:1620] T a2426b38ca0341339d254b225053f261 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:33.494575 29042 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=2374ce94fd564590bf63536fb6960027] alter complete (version 2)
I20240502 20:46:33.596325 29040 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:33.605276 6974 tablet.cc:1722] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:33.608528 6975 tablet.cc:1722] T a2426b38ca0341339d254b225053f261 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:33.609328 6976 tablet.cc:1722] T a2426b38ca0341339d254b225053f261 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:33.618963 6237 consensus_queue.cc:1035] T 3126707d1c8c4c1786da5bfdfce53eb0 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:33.619504 29034 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=2374ce94fd564590bf63536fb6960027] alter complete (version 3)
I20240502 20:46:33.636255 6234 consensus_queue.cc:1035] T 3126707d1c8c4c1786da5bfdfce53eb0 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:33.695614 29295 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:33.702872 31602 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:33.703943 29425 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 20:46:33.705381 29162 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 20:46:33.707759 29557 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:33.708473 31868 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 20:46:33.711774 29040 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_54"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:33.718052 31475 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:33.718636 29162 tablet_service.cc:1467] Processing CreateTablet for tablet fe41756b5dc74f7f9105e002b8bc9c84 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=8a4871862a4c45b7829707b34b9888c7]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:33.718907 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe41756b5dc74f7f9105e002b8bc9c84. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:33.719367 31475 tablet_service.cc:1467] Processing CreateTablet for tablet 4afde5c1bf754a819f84bcc41212617d (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=8a4871862a4c45b7829707b34b9888c7]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:33.719602 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4afde5c1bf754a819f84bcc41212617d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:33.719534 29295 tablet_service.cc:1467] Processing CreateTablet for tablet fe41756b5dc74f7f9105e002b8bc9c84 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=8a4871862a4c45b7829707b34b9888c7]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:33.719789 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe41756b5dc74f7f9105e002b8bc9c84. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:33.721341 29163 tablet_service.cc:1467] Processing CreateTablet for tablet 4afde5c1bf754a819f84bcc41212617d (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=8a4871862a4c45b7829707b34b9888c7]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:33.721575 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4afde5c1bf754a819f84bcc41212617d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:33.722010 31474 tablet_service.cc:1467] Processing CreateTablet for tablet 5993321dc8f54f14a89fa2ba4a44ccc7 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=8a4871862a4c45b7829707b34b9888c7]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:33.722213 31474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5993321dc8f54f14a89fa2ba4a44ccc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:33.723698 31602 tablet_service.cc:1467] Processing CreateTablet for tablet fe41756b5dc74f7f9105e002b8bc9c84 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=8a4871862a4c45b7829707b34b9888c7]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:33.723933 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe41756b5dc74f7f9105e002b8bc9c84. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:33.726150 29034 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:33.726258 29034 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:33.727085 31605 tablet_service.cc:1467] Processing CreateTablet for tablet 4afde5c1bf754a819f84bcc41212617d (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=8a4871862a4c45b7829707b34b9888c7]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:33.727308 31605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4afde5c1bf754a819f84bcc41212617d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:33.727581 29425 tablet_service.cc:1467] Processing CreateTablet for tablet 5993321dc8f54f14a89fa2ba4a44ccc7 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=8a4871862a4c45b7829707b34b9888c7]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:33.727820 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5993321dc8f54f14a89fa2ba4a44ccc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:33.731667 7038 tablet_bootstrap.cc:492] T fe41756b5dc74f7f9105e002b8bc9c84 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:33.732503 7038 tablet_bootstrap.cc:654] T fe41756b5dc74f7f9105e002b8bc9c84 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:33.736673 29034 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet a2426b38ca0341339d254b225053f261
I20240502 20:46:33.737165 7038 tablet_bootstrap.cc:492] T fe41756b5dc74f7f9105e002b8bc9c84 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:33.737308 31868 tablet_service.cc:1514] Processing DeleteTablet for tablet a2426b38ca0341339d254b225053f261 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:33 UTC) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:33.737929 31474 tablet_service.cc:1514] Processing DeleteTablet for tablet a2426b38ca0341339d254b225053f261 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:33 UTC) from {username='slave'} at 127.0.0.1:40276
I20240502 20:46:33.737974 7038 ts_tablet_manager.cc:1367] T fe41756b5dc74f7f9105e002b8bc9c84 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 20:46:33.738410 7038 raft_consensus.cc:348] T fe41756b5dc74f7f9105e002b8bc9c84 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:33.739740 7039 tablet_bootstrap.cc:492] T 4afde5c1bf754a819f84bcc41212617d P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:33.740998 7039 tablet_bootstrap.cc:654] T 4afde5c1bf754a819f84bcc41212617d P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:33.742866 7038 raft_consensus.cc:374] T fe41756b5dc74f7f9105e002b8bc9c84 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:33.743280 7038 raft_consensus.cc:729] T fe41756b5dc74f7f9105e002b8bc9c84 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:33.743484 7038 consensus_queue.cc:260] T fe41756b5dc74f7f9105e002b8bc9c84 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:33.743825 7038 ts_tablet_manager.cc:1398] T fe41756b5dc74f7f9105e002b8bc9c84 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20240502 20:46:33.744484 7038 tablet_bootstrap.cc:492] T 4afde5c1bf754a819f84bcc41212617d P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:33.745360 7038 tablet_bootstrap.cc:654] T 4afde5c1bf754a819f84bcc41212617d P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:33.748071 7044 tablet_replica.cc:330] T a2426b38ca0341339d254b225053f261 P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:33.748230 7044 raft_consensus.cc:2229] T a2426b38ca0341339d254b225053f261 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:33.748431 7044 raft_consensus.cc:2258] T a2426b38ca0341339d254b225053f261 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:33.749663 7044 ts_tablet_manager.cc:1875] T a2426b38ca0341339d254b225053f261 P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:33.750247 31736 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:33.751190 31602 tablet_service.cc:1467] Processing CreateTablet for tablet 5993321dc8f54f14a89fa2ba4a44ccc7 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=8a4871862a4c45b7829707b34b9888c7]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:33.751461 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5993321dc8f54f14a89fa2ba4a44ccc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:33.753448 7044 ts_tablet_manager.cc:1888] T a2426b38ca0341339d254b225053f261 P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:33.753536 7044 log.cc:1196] T a2426b38ca0341339d254b225053f261 P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/a2426b38ca0341339d254b225053f261
I20240502 20:46:33.753815 7044 ts_tablet_manager.cc:1909] T a2426b38ca0341339d254b225053f261 P e6b60803d0764d228c863e2da65e3691: Deleting consensus metadata
I20240502 20:46:33.754606 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet a2426b38ca0341339d254b225053f261 (table rebalancer_dynamic_table_new_name [id=2374ce94fd564590bf63536fb6960027]) successfully deleted
I20240502 20:46:33.754827 7038 tablet_bootstrap.cc:492] T 4afde5c1bf754a819f84bcc41212617d P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:33.754917 29425 tablet_service.cc:1514] Processing DeleteTablet for tablet a2426b38ca0341339d254b225053f261 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:33 UTC) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:33.754989 7038 ts_tablet_manager.cc:1367] T 4afde5c1bf754a819f84bcc41212617d P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 20:46:33.757805 7038 raft_consensus.cc:348] T 4afde5c1bf754a819f84bcc41212617d P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:33.757982 7038 raft_consensus.cc:374] T 4afde5c1bf754a819f84bcc41212617d P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:33.758059 7038 raft_consensus.cc:729] T 4afde5c1bf754a819f84bcc41212617d P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:33.758250 7038 consensus_queue.cc:260] T 4afde5c1bf754a819f84bcc41212617d P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:33.759052 7038 ts_tablet_manager.cc:1398] T 4afde5c1bf754a819f84bcc41212617d P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:33.775521 7040 tablet_bootstrap.cc:492] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:33.775779 7041 tablet_bootstrap.cc:492] T fe41756b5dc74f7f9105e002b8bc9c84 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:33.776862 7041 tablet_bootstrap.cc:654] T fe41756b5dc74f7f9105e002b8bc9c84 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:33.777971 7041 tablet_bootstrap.cc:492] T fe41756b5dc74f7f9105e002b8bc9c84 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:33.778625 7041 ts_tablet_manager.cc:1367] T fe41756b5dc74f7f9105e002b8bc9c84 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:33.778654 7042 tablet_bootstrap.cc:492] T 5993321dc8f54f14a89fa2ba4a44ccc7 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:33.779155 7041 raft_consensus.cc:348] T fe41756b5dc74f7f9105e002b8bc9c84 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:33.779131 7045 tablet_replica.cc:330] T a2426b38ca0341339d254b225053f261 P 2a9c1af8cf814030bde56b67ca5c2fbe: stopping tablet replica
I20240502 20:46:33.779661 7045 raft_consensus.cc:2229] T a2426b38ca0341339d254b225053f261 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:33.779839 7045 raft_consensus.cc:2258] T a2426b38ca0341339d254b225053f261 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:33.780320 7048 tablet_replica.cc:330] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:33.780470 7048 raft_consensus.cc:2229] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:33.780665 7048 raft_consensus.cc:2258] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:33.781970 7041 raft_consensus.cc:374] T fe41756b5dc74f7f9105e002b8bc9c84 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:33.782081 7041 raft_consensus.cc:729] T fe41756b5dc74f7f9105e002b8bc9c84 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:33.782156 7042 tablet_bootstrap.cc:654] T 5993321dc8f54f14a89fa2ba4a44ccc7 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:33.782256 7041 consensus_queue.cc:260] T fe41756b5dc74f7f9105e002b8bc9c84 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:33.782572 7045 ts_tablet_manager.cc:1875] T a2426b38ca0341339d254b225053f261 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:33.783638 7041 ts_tablet_manager.cc:1398] T fe41756b5dc74f7f9105e002b8bc9c84 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 20:46:33.783857 7041 tablet_bootstrap.cc:492] T 4afde5c1bf754a819f84bcc41212617d P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:33.782529 7048 ts_tablet_manager.cc:1875] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:33.784848 7041 tablet_bootstrap.cc:654] T 4afde5c1bf754a819f84bcc41212617d P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:33.785660 7040 tablet_bootstrap.cc:654] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:33.786761 7048 ts_tablet_manager.cc:1888] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:33.787202 7045 ts_tablet_manager.cc:1888] T a2426b38ca0341339d254b225053f261 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:33.787335 7048 log.cc:1196] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/a2426b38ca0341339d254b225053f261
I20240502 20:46:33.787276 7045 log.cc:1196] T a2426b38ca0341339d254b225053f261 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/wals/a2426b38ca0341339d254b225053f261
I20240502 20:46:33.787711 7048 ts_tablet_manager.cc:1909] T a2426b38ca0341339d254b225053f261 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting consensus metadata
I20240502 20:46:33.788432 7045 ts_tablet_manager.cc:1909] T a2426b38ca0341339d254b225053f261 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting consensus metadata
I20240502 20:46:33.789003 7040 tablet_bootstrap.cc:492] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:33.789095 7040 ts_tablet_manager.cc:1367] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 20:46:33.789502 29028 catalog_manager.cc:4830] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): tablet a2426b38ca0341339d254b225053f261 (table rebalancer_dynamic_table_new_name [id=2374ce94fd564590bf63536fb6960027]) successfully deleted
I20240502 20:46:33.789544 7040 raft_consensus.cc:348] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:33.789649 7040 raft_consensus.cc:374] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:33.789680 7040 raft_consensus.cc:729] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:33.789803 7040 consensus_queue.cc:260] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:33.790074 7040 ts_tablet_manager.cc:1398] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:33.791448 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet a2426b38ca0341339d254b225053f261 (table rebalancer_dynamic_table_new_name [id=2374ce94fd564590bf63536fb6960027]) successfully deleted
I20240502 20:46:33.792351 7039 tablet_bootstrap.cc:492] T 4afde5c1bf754a819f84bcc41212617d P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:33.792435 7041 tablet_bootstrap.cc:492] T 4afde5c1bf754a819f84bcc41212617d P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:33.792450 7039 ts_tablet_manager.cc:1367] T 4afde5c1bf754a819f84bcc41212617d P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.053s user 0.002s sys 0.000s
I20240502 20:46:33.792510 7041 ts_tablet_manager.cc:1367] T 4afde5c1bf754a819f84bcc41212617d P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 20:46:33.792896 7041 raft_consensus.cc:348] T 4afde5c1bf754a819f84bcc41212617d P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:33.793004 7041 raft_consensus.cc:374] T 4afde5c1bf754a819f84bcc41212617d P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:33.792943 7039 raft_consensus.cc:348] T 4afde5c1bf754a819f84bcc41212617d P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:33.793033 7041 raft_consensus.cc:729] T 4afde5c1bf754a819f84bcc41212617d P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:33.793049 7039 raft_consensus.cc:374] T 4afde5c1bf754a819f84bcc41212617d P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:33.793083 7039 raft_consensus.cc:729] T 4afde5c1bf754a819f84bcc41212617d P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:33.793135 7041 consensus_queue.cc:260] T 4afde5c1bf754a819f84bcc41212617d P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:33.793193 7039 consensus_queue.cc:260] T 4afde5c1bf754a819f84bcc41212617d P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:33.793480 7039 ts_tablet_manager.cc:1398] T 4afde5c1bf754a819f84bcc41212617d P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:33.793666 7039 tablet_bootstrap.cc:492] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:33.794034 7041 ts_tablet_manager.cc:1398] T 4afde5c1bf754a819f84bcc41212617d P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:33.794183 7041 tablet_bootstrap.cc:492] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:33.794685 7039 tablet_bootstrap.cc:654] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:33.795008 7041 tablet_bootstrap.cc:654] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:33.788849 7042 tablet_bootstrap.cc:492] T 5993321dc8f54f14a89fa2ba4a44ccc7 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:33.795205 7042 ts_tablet_manager.cc:1367] T 5993321dc8f54f14a89fa2ba4a44ccc7 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 20:46:33.795668 7042 raft_consensus.cc:348] T 5993321dc8f54f14a89fa2ba4a44ccc7 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:33.795770 7042 raft_consensus.cc:374] T 5993321dc8f54f14a89fa2ba4a44ccc7 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:33.795800 7042 raft_consensus.cc:729] T 5993321dc8f54f14a89fa2ba4a44ccc7 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:33.796360 7041 tablet_bootstrap.cc:492] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:33.796450 7041 ts_tablet_manager.cc:1367] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:33.796838 7041 raft_consensus.cc:348] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:33.796936 7041 raft_consensus.cc:374] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:33.796968 7041 raft_consensus.cc:729] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:33.797070 7041 consensus_queue.cc:260] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:33.797164 7039 tablet_bootstrap.cc:492] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:33.797235 7039 ts_tablet_manager.cc:1367] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:33.797319 7041 ts_tablet_manager.cc:1398] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:33.797644 7039 raft_consensus.cc:348] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:33.797750 7039 raft_consensus.cc:374] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:33.797780 7039 raft_consensus.cc:729] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:33.797888 7039 consensus_queue.cc:260] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:33.798164 7039 ts_tablet_manager.cc:1398] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:33.799427 7042 consensus_queue.cc:260] T 5993321dc8f54f14a89fa2ba4a44ccc7 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:33.799814 7042 ts_tablet_manager.cc:1398] T 5993321dc8f54f14a89fa2ba4a44ccc7 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 20:46:33.838351 29036 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:33.844477 31474 tablet_service.cc:1467] Processing CreateTablet for tablet bfc12153a6f3430f9a3a0e8d82c4f5e3 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a1d2bddd67fc4c3dadae41a3001d2eb4]), partition=
I20240502 20:46:33.844501 31868 tablet_service.cc:1467] Processing CreateTablet for tablet bfc12153a6f3430f9a3a0e8d82c4f5e3 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a1d2bddd67fc4c3dadae41a3001d2eb4]), partition=
I20240502 20:46:33.844756 31474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfc12153a6f3430f9a3a0e8d82c4f5e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:33.844761 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfc12153a6f3430f9a3a0e8d82c4f5e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:33.846824 7039 tablet_bootstrap.cc:492] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:33.847894 7039 tablet_bootstrap.cc:654] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:33.848230 31736 tablet_service.cc:1467] Processing CreateTablet for tablet bfc12153a6f3430f9a3a0e8d82c4f5e3 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a1d2bddd67fc4c3dadae41a3001d2eb4]), partition=
I20240502 20:46:33.848485 31736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfc12153a6f3430f9a3a0e8d82c4f5e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:33.849807 7039 tablet_bootstrap.cc:492] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:33.849884 7039 ts_tablet_manager.cc:1367] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:33.850311 7039 raft_consensus.cc:348] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:33.850404 7039 raft_consensus.cc:374] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:33.850441 7039 raft_consensus.cc:729] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:33.850576 7039 consensus_queue.cc:260] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:33.850831 7039 ts_tablet_manager.cc:1398] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:33.850864 7065 tablet_bootstrap.cc:492] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:33.851822 7065 tablet_bootstrap.cc:654] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:33.854351 7067 tablet_bootstrap.cc:492] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:33.855201 7067 tablet_bootstrap.cc:654] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:33.857355 7067 tablet_bootstrap.cc:492] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:33.857452 7067 ts_tablet_manager.cc:1367] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:33.857411 31887 consensus_queue.cc:237] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:33.857939 7067 raft_consensus.cc:348] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:33.858057 7067 raft_consensus.cc:374] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:33.858124 7067 raft_consensus.cc:729] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:33.858283 7067 consensus_queue.cc:260] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:33.858753 7067 ts_tablet_manager.cc:1398] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:33.859272 31625 raft_consensus.cc:1261] T 3eb48ef3d5404e5e9c302b904b5df2cd P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:33.861696 31494 raft_consensus.cc:1261] T 3eb48ef3d5404e5e9c302b904b5df2cd P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:33.861822 7024 consensus_queue.cc:1035] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 20:46:33.862442 31823 consensus_peers.cc:485] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691 -> Peer 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): Couldn't send request to peer 09258ac095664fa99caf0707262cde13. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3eb48ef3d5404e5e9c302b904b5df2cd. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:33.862787 7024 consensus_queue.cc:1035] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:33.865161 7065 tablet_bootstrap.cc:492] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:33.865265 7065 ts_tablet_manager.cc:1367] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 20:46:33.865746 7065 raft_consensus.cc:348] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:33.865851 7065 raft_consensus.cc:374] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:33.865886 7065 raft_consensus.cc:729] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:33.866039 7065 consensus_queue.cc:260] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:33.866348 7065 ts_tablet_manager.cc:1398] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:33.866637 7024 raft_consensus.cc:2932] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:33.868501 31625 raft_consensus.cc:2932] T 3eb48ef3d5404e5e9c302b904b5df2cd P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:33.869817 31494 raft_consensus.cc:2932] T 3eb48ef3d5404e5e9c302b904b5df2cd P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:33.873850 29039 catalog_manager.cc:5484] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691 reported cstate change: config changed from index -1 to 2, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) added. New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:33.898790 31887 consensus_queue.cc:237] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:33.900141 31756 raft_consensus.cc:1261] T 21eab607fca143b8a165d777b50238e1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 20:46:33.900274 31824 consensus_peers.cc:485] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691 -> Peer 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): Couldn't send request to peer 7796ea3fba294d17a34386873969431d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 21eab607fca143b8a165d777b50238e1. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:33.900280 29574 raft_consensus.cc:1261] T 21eab607fca143b8a165d777b50238e1 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:33.900563 7076 consensus_queue.cc:1035] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:33.900764 6850 consensus_queue.cc:1035] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:33.902652 7024 raft_consensus.cc:2932] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:33.903098 31756 raft_consensus.cc:2932] T 21eab607fca143b8a165d777b50238e1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:33.904392 29574 raft_consensus.cc:2932] T 21eab607fca143b8a165d777b50238e1 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:33.906813 29034 catalog_manager.cc:5484] T 21eab607fca143b8a165d777b50238e1 P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: config changed from index -1 to 2, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) added. New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:33.913168 7076 raft_consensus.cc:482] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:33.913255 7076 raft_consensus.cc:504] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:33.913651 7076 leader_election.cc:288] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:33.913936 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfc12153a6f3430f9a3a0e8d82c4f5e3" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:33.914079 31494 raft_consensus.cc:2454] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:33.914068 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfc12153a6f3430f9a3a0e8d82c4f5e3" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:33.914177 31756 raft_consensus.cc:2454] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:33.914425 31821 leader_election.cc:302] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e6b60803d0764d228c863e2da65e3691, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:33.914623 7076 raft_consensus.cc:2781] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:33.914675 7076 raft_consensus.cc:482] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:33.914706 7076 raft_consensus.cc:3037] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:33.915400 7076 raft_consensus.cc:504] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:33.915728 7076 leader_election.cc:288] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Requested vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:33.915987 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfc12153a6f3430f9a3a0e8d82c4f5e3" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:33.916116 31495 raft_consensus.cc:3037] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:33.916105 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfc12153a6f3430f9a3a0e8d82c4f5e3" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:33.916201 31756 raft_consensus.cc:3037] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:33.916966 31756 raft_consensus.cc:2454] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:33.917053 31495 raft_consensus.cc:2454] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:33.917215 31821 leader_election.cc:302] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e6b60803d0764d228c863e2da65e3691, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:33.917403 7076 raft_consensus.cc:2781] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:33.917496 7076 raft_consensus.cc:686] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Becoming Leader. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Running, Role: LEADER
I20240502 20:46:33.917615 7076 consensus_queue.cc:237] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:33.920557 29040 catalog_manager.cc:5484] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6b60803d0764d228c863e2da65e3691 (127.22.45.72). New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:33.932268 29444 consensus_queue.cc:237] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:33.933626 31495 raft_consensus.cc:1261] T 60e8272de78a4c94b77408c4d1d5f60e P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:33.933626 31887 raft_consensus.cc:1261] T 60e8272de78a4c94b77408c4d1d5f60e P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 20:46:33.933643 29380 consensus_peers.cc:485] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60e8272de78a4c94b77408c4d1d5f60e. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:33.934000 6789 consensus_queue.cc:1035] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:33.934196 6234 consensus_queue.cc:1035] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:33.935782 7022 raft_consensus.cc:482] T 4afde5c1bf754a819f84bcc41212617d P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:33.935868 7022 raft_consensus.cc:504] T 4afde5c1bf754a819f84bcc41212617d P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:33.935891 6234 raft_consensus.cc:2932] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:33.936261 7022 leader_election.cc:288] T 4afde5c1bf754a819f84bcc41212617d P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:33.936611 31495 raft_consensus.cc:2932] T 60e8272de78a4c94b77408c4d1d5f60e P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:33.937824 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4afde5c1bf754a819f84bcc41212617d" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:33.937955 31625 raft_consensus.cc:2454] T 4afde5c1bf754a819f84bcc41212617d P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:33.937847 31887 raft_consensus.cc:2932] T 60e8272de78a4c94b77408c4d1d5f60e P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:33.938295 31429 leader_election.cc:302] T 4afde5c1bf754a819f84bcc41212617d P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, 315915efcf2b413ca7f9d874338e1b2d; no voters:
I20240502 20:46:33.938838 6616 raft_consensus.cc:2781] T 4afde5c1bf754a819f84bcc41212617d P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:33.938896 6616 raft_consensus.cc:482] T 4afde5c1bf754a819f84bcc41212617d P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:33.938956 6616 raft_consensus.cc:3037] T 4afde5c1bf754a819f84bcc41212617d P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:33.938999 29040 catalog_manager.cc:5484] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index -1 to 2, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:33.939847 6616 raft_consensus.cc:504] T 4afde5c1bf754a819f84bcc41212617d P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:33.940165 6616 leader_election.cc:288] T 4afde5c1bf754a819f84bcc41212617d P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:33.941699 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4afde5c1bf754a819f84bcc41212617d" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:33.941733 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4afde5c1bf754a819f84bcc41212617d" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:33.941820 29175 raft_consensus.cc:2454] T 4afde5c1bf754a819f84bcc41212617d P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:33.942351 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4afde5c1bf754a819f84bcc41212617d" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:33.942490 31625 raft_consensus.cc:3037] T 4afde5c1bf754a819f84bcc41212617d P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:33.943408 31625 raft_consensus.cc:2454] T 4afde5c1bf754a819f84bcc41212617d P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:33.943682 31429 leader_election.cc:302] T 4afde5c1bf754a819f84bcc41212617d P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, 315915efcf2b413ca7f9d874338e1b2d; no voters: 7796ea3fba294d17a34386873969431d
I20240502 20:46:33.943840 6616 raft_consensus.cc:2781] T 4afde5c1bf754a819f84bcc41212617d P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:33.943914 6616 raft_consensus.cc:686] T 4afde5c1bf754a819f84bcc41212617d P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Becoming Leader. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Running, Role: LEADER
I20240502 20:46:33.944023 6616 consensus_queue.cc:237] T 4afde5c1bf754a819f84bcc41212617d P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:33.945951 6980 raft_consensus.cc:482] T fe41756b5dc74f7f9105e002b8bc9c84 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:33.946038 6980 raft_consensus.cc:504] T fe41756b5dc74f7f9105e002b8bc9c84 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:33.946425 6980 leader_election.cc:288] T fe41756b5dc74f7f9105e002b8bc9c84 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:33.946687 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe41756b5dc74f7f9105e002b8bc9c84" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:33.946806 29182 raft_consensus.cc:2454] T fe41756b5dc74f7f9105e002b8bc9c84 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:33.947273 31562 leader_election.cc:302] T fe41756b5dc74f7f9105e002b8bc9c84 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:33.947844 29313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe41756b5dc74f7f9105e002b8bc9c84" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:33.947975 29313 raft_consensus.cc:2454] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:33.948638 5804 raft_consensus.cc:2781] T fe41756b5dc74f7f9105e002b8bc9c84 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:33.948701 5804 raft_consensus.cc:482] T fe41756b5dc74f7f9105e002b8bc9c84 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:33.948748 5804 raft_consensus.cc:3037] T fe41756b5dc74f7f9105e002b8bc9c84 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:33.949561 5804 raft_consensus.cc:504] T fe41756b5dc74f7f9105e002b8bc9c84 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:33.949651 29034 catalog_manager.cc:5484] T 4afde5c1bf754a819f84bcc41212617d P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69). New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:33.949894 5804 leader_election.cc:288] T fe41756b5dc74f7f9105e002b8bc9c84 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:33.949983 5804 raft_consensus.cc:482] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:33.950038 5804 raft_consensus.cc:504] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:33.950580 5804 leader_election.cc:288] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:33.950569 29313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe41756b5dc74f7f9105e002b8bc9c84" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:33.950665 29313 raft_consensus.cc:3037] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:33.950820 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe41756b5dc74f7f9105e002b8bc9c84" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:33.950907 29182 raft_consensus.cc:3037] T fe41756b5dc74f7f9105e002b8bc9c84 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:33.951588 29313 raft_consensus.cc:2454] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:33.951823 29182 raft_consensus.cc:2454] T fe41756b5dc74f7f9105e002b8bc9c84 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:33.952114 31562 leader_election.cc:302] T fe41756b5dc74f7f9105e002b8bc9c84 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:33.952272 6980 raft_consensus.cc:2781] T fe41756b5dc74f7f9105e002b8bc9c84 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:33.952283 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5993321dc8f54f14a89fa2ba4a44ccc7" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:33.952329 6980 raft_consensus.cc:686] T fe41756b5dc74f7f9105e002b8bc9c84 P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Becoming Leader. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Running, Role: LEADER
I20240502 20:46:33.952384 31495 raft_consensus.cc:2454] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:33.952440 6980 consensus_queue.cc:237] T fe41756b5dc74f7f9105e002b8bc9c84 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:33.952474 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5993321dc8f54f14a89fa2ba4a44ccc7" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:33.952577 29444 raft_consensus.cc:2454] T 5993321dc8f54f14a89fa2ba4a44ccc7 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 0.
I20240502 20:46:33.954573 29039 catalog_manager.cc:5484] T fe41756b5dc74f7f9105e002b8bc9c84 P 315915efcf2b413ca7f9d874338e1b2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70). New cstate: current_term: 1 leader_uuid: "315915efcf2b413ca7f9d874338e1b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:33.956028 31561 leader_election.cc:302] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, 315915efcf2b413ca7f9d874338e1b2d; no voters:
I20240502 20:46:33.956463 5804 raft_consensus.cc:2781] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:33.956518 5804 raft_consensus.cc:482] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:33.956549 5804 raft_consensus.cc:3037] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:33.957173 5804 raft_consensus.cc:504] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:33.957470 5804 leader_election.cc:288] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Requested vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:33.957783 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5993321dc8f54f14a89fa2ba4a44ccc7" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:33.957780 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5993321dc8f54f14a89fa2ba4a44ccc7" candidate_uuid: "315915efcf2b413ca7f9d874338e1b2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:33.957870 31494 raft_consensus.cc:3037] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:33.957875 29444 raft_consensus.cc:3037] T 5993321dc8f54f14a89fa2ba4a44ccc7 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:33.958552 29444 raft_consensus.cc:2454] T 5993321dc8f54f14a89fa2ba4a44ccc7 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:33.958736 31494 raft_consensus.cc:2454] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 315915efcf2b413ca7f9d874338e1b2d in term 1.
I20240502 20:46:33.958822 31561 leader_election.cc:302] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 315915efcf2b413ca7f9d874338e1b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:33.959594 5804 raft_consensus.cc:2781] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:33.959676 5804 raft_consensus.cc:686] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Becoming Leader. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Running, Role: LEADER
I20240502 20:46:33.959789 5804 consensus_queue.cc:237] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:33.961705 29039 catalog_manager.cc:5484] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 315915efcf2b413ca7f9d874338e1b2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70). New cstate: current_term: 1 leader_uuid: "315915efcf2b413ca7f9d874338e1b2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:33.979719 31756 consensus_queue.cc:237] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:33.981034 31494 raft_consensus.cc:1261] T 75ace61b6538462b9bd5c0820eeacca9 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Refusing update from remote peer e7ab6b9053664ee5aa71c7978e26fe6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:33.981060 31887 raft_consensus.cc:1261] T 75ace61b6538462b9bd5c0820eeacca9 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer e7ab6b9053664ee5aa71c7978e26fe6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 20:46:33.981243 31692 consensus_peers.cc:485] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): Couldn't send request to peer dbcdc6ac635b44b185c3eba63f66e9d6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 75ace61b6538462b9bd5c0820eeacca9. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:33.981475 3289 consensus_queue.cc:1035] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:33.981645 7136 consensus_queue.cc:1035] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:33.983379 3289 raft_consensus.cc:2932] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:33.983817 31887 raft_consensus.cc:2932] T 75ace61b6538462b9bd5c0820eeacca9 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:33.985441 31494 raft_consensus.cc:2932] T 75ace61b6538462b9bd5c0820eeacca9 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:33.986284 29034 catalog_manager.cc:5484] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: config changed from index -1 to 2, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) added. New cstate: current_term: 1 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:34.006306 29444 consensus_queue.cc:237] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:34.009322 31625 raft_consensus.cc:1261] T db9c5ce233da4b1a9ce70ae04aef7658 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
W20240502 20:46:34.009568 29380 consensus_peers.cc:485] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): Couldn't send request to peer e6b60803d0764d228c863e2da65e3691. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: db9c5ce233da4b1a9ce70ae04aef7658. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:34.009891 6237 consensus_queue.cc:1035] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 20:46:34.012058 29574 raft_consensus.cc:1261] T db9c5ce233da4b1a9ce70ae04aef7658 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 20:46:34.012461 6789 consensus_queue.cc:1035] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 20:46:34.015093 6237 raft_consensus.cc:2932] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:34.017683 31624 raft_consensus.cc:2932] T db9c5ce233da4b1a9ce70ae04aef7658 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:34.018741 29036 catalog_manager.cc:5484] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index 5 to 6, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:34.020076 29574 raft_consensus.cc:2932] T db9c5ce233da4b1a9ce70ae04aef7658 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:34.024718 29036 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:34.028882 31756 raft_consensus.cc:1261] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:34.029407 31494 raft_consensus.cc:1261] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:34.029361 7024 consensus_queue.cc:1035] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:34.029966 7024 consensus_queue.cc:1035] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:34.032316 29036 catalog_manager.cc:5576] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787) does not have the latest schema for tablet bfc12153a6f3430f9a3a0e8d82c4f5e3 (table rebalancer_dynamic_table [id=a1d2bddd67fc4c3dadae41a3001d2eb4]). Expected version 1 got 0
I20240502 20:46:34.032289 6976 tablet.cc:1722] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:34.032593 6976 tablet.cc:1620] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:34.035245 29039 catalog_manager.cc:5576] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887) does not have the latest schema for tablet bfc12153a6f3430f9a3a0e8d82c4f5e3 (table rebalancer_dynamic_table [id=a1d2bddd67fc4c3dadae41a3001d2eb4]). Expected version 1 got 0
I20240502 20:46:34.037786 6975 tablet.cc:1722] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:34.038079 6975 tablet.cc:1620] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:34.040071 29036 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=a1d2bddd67fc4c3dadae41a3001d2eb4] alter complete (version 1)
I20240502 20:46:34.040191 29034 catalog_manager.cc:5576] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887) does not have the latest schema for tablet bfc12153a6f3430f9a3a0e8d82c4f5e3 (table rebalancer_dynamic_table [id=a1d2bddd67fc4c3dadae41a3001d2eb4]). Expected version 1 got 0
I20240502 20:46:34.040946 7162 mvcc.cc:205] Tried to move back new op lower bound from 7023340724339228672 to 7023340723887906816. Current Snapshot: MvccSnapshot[applied={T|T < 7023340724339228672}]
I20240502 20:46:34.041350 7170 tablet.cc:1722] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:34.041709 7170 tablet.cc:1620] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:34.048538 29315 consensus_queue.cc:237] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:34.049798 29574 raft_consensus.cc:1261] T 370bf1bcfb3343c488954ece2488b4c2 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 20:46:34.050050 29248 consensus_peers.cc:485] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 -> Peer e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): Couldn't send request to peer e7ab6b9053664ee5aa71c7978e26fe6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 370bf1bcfb3343c488954ece2488b4c2. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:34.050248 6454 consensus_queue.cc:1035] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:34.050688 29182 raft_consensus.cc:1261] T 370bf1bcfb3343c488954ece2488b4c2 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:34.051052 7177 consensus_queue.cc:1035] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:34.052631 6454 raft_consensus.cc:2932] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:34.053051 29182 raft_consensus.cc:2932] T 370bf1bcfb3343c488954ece2488b4c2 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:34.055253 29575 raft_consensus.cc:2932] T 370bf1bcfb3343c488954ece2488b4c2 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:34.055609 29034 catalog_manager.cc:5484] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 reported cstate change: config changed from index -1 to 2, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:34.073289 31756 consensus_queue.cc:237] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:34.074656 31886 raft_consensus.cc:1261] T 8705260a889841bfa515683432dfe99d P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer e7ab6b9053664ee5aa71c7978e26fe6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:34.074741 29182 raft_consensus.cc:1261] T 8705260a889841bfa515683432dfe99d P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer e7ab6b9053664ee5aa71c7978e26fe6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:34.074992 6681 consensus_queue.cc:1035] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 20:46:34.075179 31691 consensus_peers.cc:485] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8705260a889841bfa515683432dfe99d. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:34.076215 6681 consensus_queue.cc:1035] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:34.077713 6681 raft_consensus.cc:2932] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:34.078646 29175 raft_consensus.cc:2932] T 8705260a889841bfa515683432dfe99d P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:34.080246 31886 raft_consensus.cc:2932] T 8705260a889841bfa515683432dfe99d P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:34.080763 29039 catalog_manager.cc:5484] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: config changed from index -1 to 2, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New cstate: current_term: 1 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:34.098870 29315 consensus_queue.cc:237] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:34.100220 31756 raft_consensus.cc:1261] T 43fc3dcc2dd54a11becb8f16ec500df1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 20:46:34.100430 29250 consensus_peers.cc:485] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 -> Peer dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): Couldn't send request to peer dbcdc6ac635b44b185c3eba63f66e9d6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43fc3dcc2dd54a11becb8f16ec500df1. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:34.100658 7023 consensus_queue.cc:1035] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:34.100845 31886 raft_consensus.cc:1261] T 43fc3dcc2dd54a11becb8f16ec500df1 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:34.101195 7023 consensus_queue.cc:1035] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:34.102730 7177 raft_consensus.cc:2932] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:34.103101 31886 raft_consensus.cc:2932] T 43fc3dcc2dd54a11becb8f16ec500df1 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:34.105711 29036 catalog_manager.cc:5484] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 reported cstate change: config changed from index -1 to 2, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) added. New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:34.106523 31756 raft_consensus.cc:2932] T 43fc3dcc2dd54a11becb8f16ec500df1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:34.125536 29175 consensus_queue.cc:237] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:34.126859 31624 raft_consensus.cc:1261] T b28fe6bccf4f48a6a5456e6ead5cc920 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
W20240502 20:46:34.127002 29119 consensus_peers.cc:485] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d -> Peer 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): Couldn't send request to peer 2a9c1af8cf814030bde56b67ca5c2fbe. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b28fe6bccf4f48a6a5456e6ead5cc920. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:34.127019 29575 raft_consensus.cc:1261] T b28fe6bccf4f48a6a5456e6ead5cc920 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 20:46:34.127526 6633 consensus_queue.cc:1035] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 20:46:34.127833 6633 consensus_queue.cc:1035] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 20:46:34.129846 6633 raft_consensus.cc:2932] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:34.130239 31625 raft_consensus.cc:2932] T b28fe6bccf4f48a6a5456e6ead5cc920 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:34.131611 29575 raft_consensus.cc:2932] T b28fe6bccf4f48a6a5456e6ead5cc920 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:34.132822 29039 catalog_manager.cc:5484] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d reported cstate change: config changed from index 5 to 6, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:34.144673 29034 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:34.150671 6976 tablet.cc:1722] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:34.150997 6976 tablet.cc:1620] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:34.151424 7170 tablet.cc:1722] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:34.151695 7170 tablet.cc:1620] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:34.152076 6975 tablet.cc:1722] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:34.152431 6975 tablet.cc:1620] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:34.156154 29575 consensus_queue.cc:237] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:34.156963 29039 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=a1d2bddd67fc4c3dadae41a3001d2eb4] alter complete (version 2)
I20240502 20:46:34.158851 31625 raft_consensus.cc:1261] T 2bec922ff95845d1a13c27526f689e15 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:34.158879 29444 raft_consensus.cc:1261] T 2bec922ff95845d1a13c27526f689e15 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 20:46:34.159271 29512 consensus_peers.cc:485] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057 -> Peer 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): Couldn't send request to peer 09258ac095664fa99caf0707262cde13. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2bec922ff95845d1a13c27526f689e15. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:34.159282 5346 consensus_queue.cc:1035] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:34.159534 6945 consensus_queue.cc:1035] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:34.161705 5346 raft_consensus.cc:2932] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:34.162106 29444 raft_consensus.cc:2932] T 2bec922ff95845d1a13c27526f689e15 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:34.163678 31625 raft_consensus.cc:2932] T 2bec922ff95845d1a13c27526f689e15 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:34.165050 29034 catalog_manager.cc:5484] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: config changed from index -1 to 2, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) added. New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:34.185951 31625 consensus_queue.cc:237] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:34.187319 31756 raft_consensus.cc:1261] T 358e4c761223404fbf0f467f38ce9b25 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer 315915efcf2b413ca7f9d874338e1b2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:34.187482 31494 raft_consensus.cc:1261] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Refusing update from remote peer 315915efcf2b413ca7f9d874338e1b2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 20:46:34.187712 31562 consensus_peers.cc:485] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d -> Peer 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): Couldn't send request to peer 7796ea3fba294d17a34386873969431d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 358e4c761223404fbf0f467f38ce9b25. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:34.187887 5804 consensus_queue.cc:1035] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:34.188094 7276 consensus_queue.cc:1035] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:34.195132 5804 raft_consensus.cc:2932] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:34.195569 31756 raft_consensus.cc:2932] T 358e4c761223404fbf0f467f38ce9b25 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:34.197686 31494 raft_consensus.cc:2932] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:34.198870 29040 catalog_manager.cc:5484] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d reported cstate change: config changed from index -1 to 2, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) added. New cstate: current_term: 1 leader_uuid: "315915efcf2b413ca7f9d874338e1b2d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:34.224625 29575 consensus_queue.cc:237] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:34.226912 31756 raft_consensus.cc:1261] T 9acc12c9f0464e07965c05285fd334f0 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
W20240502 20:46:34.227675 29511 consensus_peers.cc:485] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9acc12c9f0464e07965c05285fd334f0. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:34.229791 6945 consensus_queue.cc:1035] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 20:46:34.230695 29444 raft_consensus.cc:1261] T 9acc12c9f0464e07965c05285fd334f0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 20:46:34.231139 6945 consensus_queue.cc:1035] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 20:46:34.232777 6945 raft_consensus.cc:2932] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:34.233175 29444 raft_consensus.cc:2932] T 9acc12c9f0464e07965c05285fd334f0 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:34.235008 31756 raft_consensus.cc:2932] T 9acc12c9f0464e07965c05285fd334f0 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:34.235976 29034 catalog_manager.cc:5484] T 9acc12c9f0464e07965c05285fd334f0 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index 5 to 6, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:34.253818 29315 consensus_queue.cc:237] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:34.255229 31756 raft_consensus.cc:1261] T d5ca50fc288f4957a2d6f04c23bd1491 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:34.255268 31886 raft_consensus.cc:1261] T d5ca50fc288f4957a2d6f04c23bd1491 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 20:46:34.255316 29249 consensus_peers.cc:485] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13 -> Peer db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): Couldn't send request to peer db17f26a73f3447b92aa9a947e3a7057. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d5ca50fc288f4957a2d6f04c23bd1491. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:34.255568 7023 consensus_queue.cc:1035] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:34.255745 7177 consensus_queue.cc:1035] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:34.257299 7023 raft_consensus.cc:2932] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:34.257699 31886 raft_consensus.cc:2932] T d5ca50fc288f4957a2d6f04c23bd1491 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:34.258416 29040 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:34.260999 29036 catalog_manager.cc:5484] T d5ca50fc288f4957a2d6f04c23bd1491 P e6b60803d0764d228c863e2da65e3691 reported cstate change: config changed from index -1 to 2, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) added. New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:34.263116 31756 raft_consensus.cc:2932] T d5ca50fc288f4957a2d6f04c23bd1491 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:34.266273 6976 tablet.cc:1722] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:34.266953 6975 tablet.cc:1722] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:34.271484 7170 tablet.cc:1722] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:34.271565 29034 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=a1d2bddd67fc4c3dadae41a3001d2eb4] alter complete (version 3)
I20240502 20:46:34.286425 29444 consensus_queue.cc:237] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:34.287848 31756 raft_consensus.cc:1261] T b7c80c3b3a5d4505b7dbbac6bab1de59 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:34.287901 31494 raft_consensus.cc:1261] T b7c80c3b3a5d4505b7dbbac6bab1de59 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 20:46:34.288049 29381 consensus_peers.cc:485] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): Couldn't send request to peer 09258ac095664fa99caf0707262cde13. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7c80c3b3a5d4505b7dbbac6bab1de59. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:34.288200 6234 consensus_queue.cc:1035] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:34.288345 6789 consensus_queue.cc:1035] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:34.290252 6789 raft_consensus.cc:2932] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:34.290719 31756 raft_consensus.cc:2932] T b7c80c3b3a5d4505b7dbbac6bab1de59 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:34.291888 31494 raft_consensus.cc:2932] T b7c80c3b3a5d4505b7dbbac6bab1de59 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:34.293390 29036 catalog_manager.cc:5484] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index -1 to 2, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) added. New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:34.309686 31494 consensus_queue.cc:237] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:34.310858 31756 raft_consensus.cc:1261] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:34.310916 31886 raft_consensus.cc:1261] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 20:46:34.311040 31431 consensus_peers.cc:485] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): Couldn't send request to peer 7796ea3fba294d17a34386873969431d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2227f1ae58a044ea955cffa8d8d8a3f2. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:34.311331 7351 consensus_queue.cc:1035] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:34.311499 6616 consensus_queue.cc:1035] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:34.313318 7351 raft_consensus.cc:2932] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:34.313733 31756 raft_consensus.cc:2932] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:34.316812 29040 catalog_manager.cc:5484] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: config changed from index -1 to 2, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) added. New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:34.316969 31886 raft_consensus.cc:2932] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 7796ea3fba294d17a34386873969431d (127.22.45.65) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:34.334371 31756 consensus_queue.cc:237] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:34.335088 7279 consensus_queue.cc:1035] T fe41756b5dc74f7f9105e002b8bc9c84 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:34.335744 29315 raft_consensus.cc:1261] T 73c2e0de3fcf455ba1fecf701d23a05f P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer e7ab6b9053664ee5aa71c7978e26fe6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:34.335747 29444 raft_consensus.cc:1261] T 73c2e0de3fcf455ba1fecf701d23a05f P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer e7ab6b9053664ee5aa71c7978e26fe6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 20:46:34.336061 31691 consensus_peers.cc:485] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 73c2e0de3fcf455ba1fecf701d23a05f. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:34.336333 6681 consensus_queue.cc:1035] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:34.336510 3289 consensus_queue.cc:1035] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:34.339668 6980 consensus_queue.cc:1035] T fe41756b5dc74f7f9105e002b8bc9c84 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:34.342243 6681 raft_consensus.cc:2932] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:34.342592 29444 raft_consensus.cc:2932] T 73c2e0de3fcf455ba1fecf701d23a05f P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:34.345019 29315 raft_consensus.cc:2932] T 73c2e0de3fcf455ba1fecf701d23a05f P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:34.346426 29040 catalog_manager.cc:5484] T 73c2e0de3fcf455ba1fecf701d23a05f P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index -1 to 2, VOTER 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) added. New cstate: current_term: 1 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:34.367096 29175 consensus_queue.cc:237] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:34.368686 29315 raft_consensus.cc:1261] T c8dc196b2d4e48b195524bc79c07b9eb P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:34.368705 29575 raft_consensus.cc:1261] T c8dc196b2d4e48b195524bc79c07b9eb P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:34.369123 6777 consensus_queue.cc:1035] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 20:46:34.369336 29119 consensus_peers.cc:485] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d -> Peer 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): Couldn't send request to peer 2a9c1af8cf814030bde56b67ca5c2fbe. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c8dc196b2d4e48b195524bc79c07b9eb. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:34.369449 6777 consensus_queue.cc:1035] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:34.371073 7239 raft_consensus.cc:2932] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:34.371490 29575 raft_consensus.cc:2932] T c8dc196b2d4e48b195524bc79c07b9eb P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:34.373122 29315 raft_consensus.cc:2932] T c8dc196b2d4e48b195524bc79c07b9eb P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:34.374330 29034 catalog_manager.cc:5484] T c8dc196b2d4e48b195524bc79c07b9eb P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: config changed from index -1 to 2, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) added. New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:34.375640 6980 consensus_queue.cc:1035] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:34.375873 29039 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:34.375979 29039 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:34.380825 7279 consensus_queue.cc:1035] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 315915efcf2b413ca7f9d874338e1b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:34.382478 29039 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet bfc12153a6f3430f9a3a0e8d82c4f5e3
I20240502 20:46:34.383507 31736 tablet_service.cc:1514] Processing DeleteTablet for tablet bfc12153a6f3430f9a3a0e8d82c4f5e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:34 UTC) from {username='slave'} at 127.0.0.1:33864
I20240502 20:46:34.383721 31868 tablet_service.cc:1514] Processing DeleteTablet for tablet bfc12153a6f3430f9a3a0e8d82c4f5e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:34 UTC) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:34.383795 7402 tablet_replica.cc:330] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e7ab6b9053664ee5aa71c7978e26fe6a: stopping tablet replica
I20240502 20:46:34.383926 7402 raft_consensus.cc:2229] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:34.384054 7402 raft_consensus.cc:2258] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:34.384621 7402 ts_tablet_manager.cc:1875] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:34.385241 31474 tablet_service.cc:1514] Processing DeleteTablet for tablet bfc12153a6f3430f9a3a0e8d82c4f5e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:34 UTC) from {username='slave'} at 127.0.0.1:40276
I20240502 20:46:34.387459 7402 ts_tablet_manager.cc:1888] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:34.387583 7402 log.cc:1196] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/wals/bfc12153a6f3430f9a3a0e8d82c4f5e3
I20240502 20:46:34.387874 7402 ts_tablet_manager.cc:1909] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting consensus metadata
I20240502 20:46:34.388195 7403 tablet_replica.cc:330] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:34.388425 7403 raft_consensus.cc:2229] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:34.388708 7403 raft_consensus.cc:2258] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:34.389089 29026 catalog_manager.cc:4830] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): tablet bfc12153a6f3430f9a3a0e8d82c4f5e3 (table rebalancer_dynamic_table_new_name [id=a1d2bddd67fc4c3dadae41a3001d2eb4]) successfully deleted
I20240502 20:46:34.389448 7403 ts_tablet_manager.cc:1875] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:34.390645 7404 tablet_replica.cc:330] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P 2a9c1af8cf814030bde56b67ca5c2fbe: stopping tablet replica
I20240502 20:46:34.390777 7404 raft_consensus.cc:2229] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:34.390874 7404 raft_consensus.cc:2258] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:34.391387 7404 ts_tablet_manager.cc:1875] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:34.391537 7403 ts_tablet_manager.cc:1888] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:34.391613 7403 log.cc:1196] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/bfc12153a6f3430f9a3a0e8d82c4f5e3
I20240502 20:46:34.391887 7403 ts_tablet_manager.cc:1909] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P e6b60803d0764d228c863e2da65e3691: Deleting consensus metadata
I20240502 20:46:34.392580 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet bfc12153a6f3430f9a3a0e8d82c4f5e3 (table rebalancer_dynamic_table_new_name [id=a1d2bddd67fc4c3dadae41a3001d2eb4]) successfully deleted
I20240502 20:46:34.393827 7404 ts_tablet_manager.cc:1888] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:34.393891 7404 log.cc:1196] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/wals/bfc12153a6f3430f9a3a0e8d82c4f5e3
I20240502 20:46:34.394230 7404 ts_tablet_manager.cc:1909] T bfc12153a6f3430f9a3a0e8d82c4f5e3 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting consensus metadata
I20240502 20:46:34.394855 29028 catalog_manager.cc:4830] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): tablet bfc12153a6f3430f9a3a0e8d82c4f5e3 (table rebalancer_dynamic_table_new_name [id=a1d2bddd67fc4c3dadae41a3001d2eb4]) successfully deleted
I20240502 20:46:34.403904 31756 consensus_queue.cc:237] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:34.405153 31887 raft_consensus.cc:1261] T 7db3d66689ef4fb2869e94448515ad12 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer e7ab6b9053664ee5aa71c7978e26fe6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:34.405308 31494 raft_consensus.cc:1261] T 7db3d66689ef4fb2869e94448515ad12 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Refusing update from remote peer e7ab6b9053664ee5aa71c7978e26fe6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:34.405534 7200 consensus_queue.cc:1035] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 20:46:34.405715 31692 consensus_peers.cc:485] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): Couldn't send request to peer 09258ac095664fa99caf0707262cde13. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7db3d66689ef4fb2869e94448515ad12. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:34.405833 7200 consensus_queue.cc:1035] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:34.407749 7136 raft_consensus.cc:2932] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:34.412703 29035 catalog_manager.cc:5484] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: config changed from index -1 to 2, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) added. New cstate: current_term: 1 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:34.413332 31494 raft_consensus.cc:2932] T 7db3d66689ef4fb2869e94448515ad12 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:34.415035 31887 raft_consensus.cc:2932] T 7db3d66689ef4fb2869e94448515ad12 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 09258ac095664fa99caf0707262cde13 (127.22.45.66) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:34.421556 7418 ts_tablet_manager.cc:902] T 3eb48ef3d5404e5e9c302b904b5df2cd P 09258ac095664fa99caf0707262cde13: Initiating tablet copy from peer e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:34.422135 7418 tablet_copy_client.cc:308] T 3eb48ef3d5404e5e9c302b904b5df2cd P 09258ac095664fa99caf0707262cde13: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.72:35207
I20240502 20:46:34.422619 31908 tablet_copy_service.cc:140] P e6b60803d0764d228c863e2da65e3691: Received BeginTabletCopySession request for tablet 3eb48ef3d5404e5e9c302b904b5df2cd from peer 09258ac095664fa99caf0707262cde13 ({username='slave'} at 127.22.45.66:55365)
I20240502 20:46:34.422719 31908 tablet_copy_service.cc:161] P e6b60803d0764d228c863e2da65e3691: Beginning new tablet copy session on tablet 3eb48ef3d5404e5e9c302b904b5df2cd from peer 09258ac095664fa99caf0707262cde13 at {username='slave'} at 127.22.45.66:55365: session id = 09258ac095664fa99caf0707262cde13-3eb48ef3d5404e5e9c302b904b5df2cd
I20240502 20:46:34.423681 31908 tablet_copy_source_session.cc:205] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:34.424559 7418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3eb48ef3d5404e5e9c302b904b5df2cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:34.426964 7418 tablet_copy_client.cc:783] T 3eb48ef3d5404e5e9c302b904b5df2cd P 09258ac095664fa99caf0707262cde13: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:34.427157 7418 tablet_copy_client.cc:647] T 3eb48ef3d5404e5e9c302b904b5df2cd P 09258ac095664fa99caf0707262cde13: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:34.428980 7418 tablet_copy_client.cc:523] T 3eb48ef3d5404e5e9c302b904b5df2cd P 09258ac095664fa99caf0707262cde13: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:34.430096 7418 tablet_bootstrap.cc:492] T 3eb48ef3d5404e5e9c302b904b5df2cd P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:34.433615 7418 tablet_bootstrap.cc:492] T 3eb48ef3d5404e5e9c302b904b5df2cd P 09258ac095664fa99caf0707262cde13: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:34.433866 7418 tablet_bootstrap.cc:492] T 3eb48ef3d5404e5e9c302b904b5df2cd P 09258ac095664fa99caf0707262cde13: Bootstrap complete.
I20240502 20:46:34.434022 7418 ts_tablet_manager.cc:1367] T 3eb48ef3d5404e5e9c302b904b5df2cd P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20240502 20:46:34.434547 7418 raft_consensus.cc:348] T 3eb48ef3d5404e5e9c302b904b5df2cd P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:34.434676 7418 raft_consensus.cc:729] T 3eb48ef3d5404e5e9c302b904b5df2cd P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:34.434813 7418 consensus_queue.cc:260] T 3eb48ef3d5404e5e9c302b904b5df2cd P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:34.435096 7418 ts_tablet_manager.cc:1398] T 3eb48ef3d5404e5e9c302b904b5df2cd P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 20:46:34.435489 31908 tablet_copy_service.cc:342] P e6b60803d0764d228c863e2da65e3691: Request end of tablet copy session 09258ac095664fa99caf0707262cde13-3eb48ef3d5404e5e9c302b904b5df2cd received from {username='slave'} at 127.22.45.66:55365
I20240502 20:46:34.435559 31908 tablet_copy_service.cc:433] P e6b60803d0764d228c863e2da65e3691: ending tablet copy session 09258ac095664fa99caf0707262cde13-3eb48ef3d5404e5e9c302b904b5df2cd on tablet 3eb48ef3d5404e5e9c302b904b5df2cd with peer 09258ac095664fa99caf0707262cde13
I20240502 20:46:34.439421 31494 consensus_queue.cc:237] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 4, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:34.440717 29175 raft_consensus.cc:1261] T 21ee3d7c6f764d0483b335da40e61efc P 7796ea3fba294d17a34386873969431d [term 2 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240502 20:46:34.440793 29315 raft_consensus.cc:1261] T 21ee3d7c6f764d0483b335da40e61efc P 09258ac095664fa99caf0707262cde13 [term 2 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240502 20:46:34.441133 7022 consensus_queue.cc:1035] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20240502 20:46:34.441318 31428 consensus_peers.cc:485] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): Couldn't send request to peer e7ab6b9053664ee5aa71c7978e26fe6a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 21ee3d7c6f764d0483b335da40e61efc. This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:34.441488 6616 consensus_queue.cc:1035] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 20:46:34.443559 7022 raft_consensus.cc:2932] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 6 to 7, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:34.444082 29175 raft_consensus.cc:2932] T 21ee3d7c6f764d0483b335da40e61efc P 7796ea3fba294d17a34386873969431d [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 6 to 7, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:34.445230 29315 raft_consensus.cc:2932] T 21ee3d7c6f764d0483b335da40e61efc P 09258ac095664fa99caf0707262cde13 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 6 to 7, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:34.447539 7443 ts_tablet_manager.cc:902] T 370bf1bcfb3343c488954ece2488b4c2 P e7ab6b9053664ee5aa71c7978e26fe6a: Initiating tablet copy from peer 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:34.447748 29035 catalog_manager.cc:5484] T 21ee3d7c6f764d0483b335da40e61efc P 7796ea3fba294d17a34386873969431d reported cstate change: config changed from index 6 to 7, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) added. New cstate: current_term: 2 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:34.448010 7443 tablet_copy_client.cc:308] T 370bf1bcfb3343c488954ece2488b4c2 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.66:40945
I20240502 20:46:34.448391 29335 tablet_copy_service.cc:140] P 09258ac095664fa99caf0707262cde13: Received BeginTabletCopySession request for tablet 370bf1bcfb3343c488954ece2488b4c2 from peer e7ab6b9053664ee5aa71c7978e26fe6a ({username='slave'} at 127.22.45.71:41427)
I20240502 20:46:34.448486 29335 tablet_copy_service.cc:161] P 09258ac095664fa99caf0707262cde13: Beginning new tablet copy session on tablet 370bf1bcfb3343c488954ece2488b4c2 from peer e7ab6b9053664ee5aa71c7978e26fe6a at {username='slave'} at 127.22.45.71:41427: session id = e7ab6b9053664ee5aa71c7978e26fe6a-370bf1bcfb3343c488954ece2488b4c2
I20240502 20:46:34.449313 29335 tablet_copy_source_session.cc:205] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:34.450151 7443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 370bf1bcfb3343c488954ece2488b4c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:34.452337 7443 tablet_copy_client.cc:783] T 370bf1bcfb3343c488954ece2488b4c2 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:34.452541 7443 tablet_copy_client.cc:647] T 370bf1bcfb3343c488954ece2488b4c2 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:34.453591 7443 tablet_copy_client.cc:523] T 370bf1bcfb3343c488954ece2488b4c2 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:34.454579 7443 tablet_bootstrap.cc:492] T 370bf1bcfb3343c488954ece2488b4c2 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:34.457763 7443 tablet_bootstrap.cc:492] T 370bf1bcfb3343c488954ece2488b4c2 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:34.457975 7443 tablet_bootstrap.cc:492] T 370bf1bcfb3343c488954ece2488b4c2 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap complete.
I20240502 20:46:34.458115 7443 ts_tablet_manager.cc:1367] T 370bf1bcfb3343c488954ece2488b4c2 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:34.458531 7443 raft_consensus.cc:348] T 370bf1bcfb3343c488954ece2488b4c2 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:34.458640 7443 raft_consensus.cc:729] T 370bf1bcfb3343c488954ece2488b4c2 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:34.458765 7443 consensus_queue.cc:260] T 370bf1bcfb3343c488954ece2488b4c2 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:34.459043 7443 ts_tablet_manager.cc:1398] T 370bf1bcfb3343c488954ece2488b4c2 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 20:46:34.459451 29335 tablet_copy_service.cc:342] P 09258ac095664fa99caf0707262cde13: Request end of tablet copy session e7ab6b9053664ee5aa71c7978e26fe6a-370bf1bcfb3343c488954ece2488b4c2 received from {username='slave'} at 127.22.45.71:41427
I20240502 20:46:34.459510 29335 tablet_copy_service.cc:433] P 09258ac095664fa99caf0707262cde13: ending tablet copy session e7ab6b9053664ee5aa71c7978e26fe6a-370bf1bcfb3343c488954ece2488b4c2 on tablet 370bf1bcfb3343c488954ece2488b4c2 with peer e7ab6b9053664ee5aa71c7978e26fe6a
I20240502 20:46:34.466065 7460 ts_tablet_manager.cc:902] T 21eab607fca143b8a165d777b50238e1 P 7796ea3fba294d17a34386873969431d: Initiating tablet copy from peer e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:34.466570 7460 tablet_copy_client.cc:308] T 21eab607fca143b8a165d777b50238e1 P 7796ea3fba294d17a34386873969431d: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.72:35207
I20240502 20:46:34.466997 31908 tablet_copy_service.cc:140] P e6b60803d0764d228c863e2da65e3691: Received BeginTabletCopySession request for tablet 21eab607fca143b8a165d777b50238e1 from peer 7796ea3fba294d17a34386873969431d ({username='slave'} at 127.22.45.65:53111)
I20240502 20:46:34.467088 31908 tablet_copy_service.cc:161] P e6b60803d0764d228c863e2da65e3691: Beginning new tablet copy session on tablet 21eab607fca143b8a165d777b50238e1 from peer 7796ea3fba294d17a34386873969431d at {username='slave'} at 127.22.45.65:53111: session id = 7796ea3fba294d17a34386873969431d-21eab607fca143b8a165d777b50238e1
I20240502 20:46:34.467828 31908 tablet_copy_source_session.cc:205] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:34.468605 7465 ts_tablet_manager.cc:902] T 75ace61b6538462b9bd5c0820eeacca9 P dbcdc6ac635b44b185c3eba63f66e9d6: Initiating tablet copy from peer e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:34.468757 7460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21eab607fca143b8a165d777b50238e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:34.469132 7465 tablet_copy_client.cc:308] T 75ace61b6538462b9bd5c0820eeacca9 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.71:36787
I20240502 20:46:34.469583 31777 tablet_copy_service.cc:140] P e7ab6b9053664ee5aa71c7978e26fe6a: Received BeginTabletCopySession request for tablet 75ace61b6538462b9bd5c0820eeacca9 from peer dbcdc6ac635b44b185c3eba63f66e9d6 ({username='slave'} at 127.22.45.67:60059)
I20240502 20:46:34.469673 31777 tablet_copy_service.cc:161] P e7ab6b9053664ee5aa71c7978e26fe6a: Beginning new tablet copy session on tablet 75ace61b6538462b9bd5c0820eeacca9 from peer dbcdc6ac635b44b185c3eba63f66e9d6 at {username='slave'} at 127.22.45.67:60059: session id = dbcdc6ac635b44b185c3eba63f66e9d6-75ace61b6538462b9bd5c0820eeacca9
I20240502 20:46:34.470536 31777 tablet_copy_source_session.cc:205] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:34.470858 7460 tablet_copy_client.cc:783] T 21eab607fca143b8a165d777b50238e1 P 7796ea3fba294d17a34386873969431d: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:34.471050 7460 tablet_copy_client.cc:647] T 21eab607fca143b8a165d777b50238e1 P 7796ea3fba294d17a34386873969431d: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:34.471375 7465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75ace61b6538462b9bd5c0820eeacca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:34.472466 7460 tablet_copy_client.cc:523] T 21eab607fca143b8a165d777b50238e1 P 7796ea3fba294d17a34386873969431d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:34.473551 7460 tablet_bootstrap.cc:492] T 21eab607fca143b8a165d777b50238e1 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:34.473666 29039 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_56"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:34.473877 7465 tablet_copy_client.cc:783] T 75ace61b6538462b9bd5c0820eeacca9 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:34.474009 7465 tablet_copy_client.cc:647] T 75ace61b6538462b9bd5c0820eeacca9 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:34.475008 7465 tablet_copy_client.cc:523] T 75ace61b6538462b9bd5c0820eeacca9 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:34.476119 7465 tablet_bootstrap.cc:492] T 75ace61b6538462b9bd5c0820eeacca9 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:34.478346 7354 consensus_queue.cc:1035] T 4afde5c1bf754a819f84bcc41212617d P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:34.480451 29425 tablet_service.cc:1467] Processing CreateTablet for tablet 8feefea4624e49309cef9ecb952bbe96 (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=214b2356a83940a0bf28fcf77f4b3c9c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:34.480686 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8feefea4624e49309cef9ecb952bbe96. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:34.480962 31474 tablet_service.cc:1467] Processing CreateTablet for tablet 5468fdb5d2a240acbda0546d54b46b74 (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=214b2356a83940a0bf28fcf77f4b3c9c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:34.481163 31474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5468fdb5d2a240acbda0546d54b46b74. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:34.482887 7460 tablet_bootstrap.cc:492] T 21eab607fca143b8a165d777b50238e1 P 7796ea3fba294d17a34386873969431d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:34.482851 29424 tablet_service.cc:1467] Processing CreateTablet for tablet 5468fdb5d2a240acbda0546d54b46b74 (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=214b2356a83940a0bf28fcf77f4b3c9c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:34.483110 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5468fdb5d2a240acbda0546d54b46b74. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:34.483114 7460 tablet_bootstrap.cc:492] T 21eab607fca143b8a165d777b50238e1 P 7796ea3fba294d17a34386873969431d: Bootstrap complete.
I20240502 20:46:34.483263 7460 ts_tablet_manager.cc:1367] T 21eab607fca143b8a165d777b50238e1 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.001s
I20240502 20:46:34.483721 31868 tablet_service.cc:1467] Processing CreateTablet for tablet 8feefea4624e49309cef9ecb952bbe96 (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=214b2356a83940a0bf28fcf77f4b3c9c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:34.483731 7460 raft_consensus.cc:348] T 21eab607fca143b8a165d777b50238e1 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:34.483832 7460 raft_consensus.cc:729] T 21eab607fca143b8a165d777b50238e1 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:34.483990 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8feefea4624e49309cef9ecb952bbe96. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:34.484134 29163 tablet_service.cc:1467] Processing CreateTablet for tablet 3443c927080841c299ca239ac9eee2be (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=214b2356a83940a0bf28fcf77f4b3c9c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:34.484354 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3443c927080841c299ca239ac9eee2be. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:34.485141 31736 tablet_service.cc:1467] Processing CreateTablet for tablet 5468fdb5d2a240acbda0546d54b46b74 (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=214b2356a83940a0bf28fcf77f4b3c9c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:34.485397 31736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5468fdb5d2a240acbda0546d54b46b74. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:34.486069 29036 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:34.486325 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 3443c927080841c299ca239ac9eee2be (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=214b2356a83940a0bf28fcf77f4b3c9c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:34.486604 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3443c927080841c299ca239ac9eee2be. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:34.486693 7460 consensus_queue.cc:260] T 21eab607fca143b8a165d777b50238e1 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:34.486995 7460 ts_tablet_manager.cc:1398] T 21eab607fca143b8a165d777b50238e1 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:34.487504 31908 tablet_copy_service.cc:342] P e6b60803d0764d228c863e2da65e3691: Request end of tablet copy session 7796ea3fba294d17a34386873969431d-21eab607fca143b8a165d777b50238e1 received from {username='slave'} at 127.22.45.65:53111
I20240502 20:46:34.487562 31908 tablet_copy_service.cc:433] P e6b60803d0764d228c863e2da65e3691: ending tablet copy session 7796ea3fba294d17a34386873969431d-21eab607fca143b8a165d777b50238e1 on tablet 21eab607fca143b8a165d777b50238e1 with peer 7796ea3fba294d17a34386873969431d
I20240502 20:46:34.482789 7472 tablet_bootstrap.cc:492] T 8feefea4624e49309cef9ecb952bbe96 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:34.488792 7481 tablet_bootstrap.cc:492] T 3443c927080841c299ca239ac9eee2be P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:34.488970 31602 tablet_service.cc:1467] Processing CreateTablet for tablet 8feefea4624e49309cef9ecb952bbe96 (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=214b2356a83940a0bf28fcf77f4b3c9c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:34.489231 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8feefea4624e49309cef9ecb952bbe96. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:34.489722 7472 tablet_bootstrap.cc:654] T 8feefea4624e49309cef9ecb952bbe96 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:34.490348 7480 tablet_bootstrap.cc:492] T 5468fdb5d2a240acbda0546d54b46b74 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:34.491273 7480 tablet_bootstrap.cc:654] T 5468fdb5d2a240acbda0546d54b46b74 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:34.491571 7481 tablet_bootstrap.cc:654] T 3443c927080841c299ca239ac9eee2be P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:34.492347 7472 tablet_bootstrap.cc:492] T 8feefea4624e49309cef9ecb952bbe96 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:34.492425 7472 ts_tablet_manager.cc:1367] T 8feefea4624e49309cef9ecb952bbe96 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 20:46:34.492746 31868 tablet_service.cc:1467] Processing CreateTablet for tablet 3443c927080841c299ca239ac9eee2be (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=214b2356a83940a0bf28fcf77f4b3c9c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:34.492960 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3443c927080841c299ca239ac9eee2be. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:34.494451 29424 tablet_service.cc:1467] Processing CreateTablet for tablet 4432691bad6b42619ef72c41e72d27a9 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=e568822f039f4f97a3afbf915c8eaeca]), partition=
I20240502 20:46:34.494663 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4432691bad6b42619ef72c41e72d27a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:34.495136 7478 tablet_bootstrap.cc:492] T 8feefea4624e49309cef9ecb952bbe96 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:34.495579 29557 tablet_service.cc:1467] Processing CreateTablet for tablet 4432691bad6b42619ef72c41e72d27a9 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=e568822f039f4f97a3afbf915c8eaeca]), partition=
I20240502 20:46:34.495820 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4432691bad6b42619ef72c41e72d27a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:34.496197 7473 tablet_bootstrap.cc:492] T 5468fdb5d2a240acbda0546d54b46b74 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:34.496515 7465 tablet_bootstrap.cc:492] T 75ace61b6538462b9bd5c0820eeacca9 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:34.496527 31736 tablet_service.cc:1467] Processing CreateTablet for tablet 4432691bad6b42619ef72c41e72d27a9 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=e568822f039f4f97a3afbf915c8eaeca]), partition=
I20240502 20:46:34.496696 31736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4432691bad6b42619ef72c41e72d27a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:34.496728 7465 tablet_bootstrap.cc:492] T 75ace61b6538462b9bd5c0820eeacca9 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap complete.
I20240502 20:46:34.496867 7465 ts_tablet_manager.cc:1367] T 75ace61b6538462b9bd5c0820eeacca9 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.000s
I20240502 20:46:34.497141 7473 tablet_bootstrap.cc:654] T 5468fdb5d2a240acbda0546d54b46b74 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:34.497401 7465 raft_consensus.cc:348] T 75ace61b6538462b9bd5c0820eeacca9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:34.497510 7465 raft_consensus.cc:729] T 75ace61b6538462b9bd5c0820eeacca9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:34.497660 7465 consensus_queue.cc:260] T 75ace61b6538462b9bd5c0820eeacca9 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:34.499379 7472 raft_consensus.cc:348] T 8feefea4624e49309cef9ecb952bbe96 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:34.499537 7472 raft_consensus.cc:374] T 8feefea4624e49309cef9ecb952bbe96 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:34.499588 29182 raft_consensus.cc:3037] T 4afde5c1bf754a819f84bcc41212617d P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:34.499606 7472 raft_consensus.cc:729] T 8feefea4624e49309cef9ecb952bbe96 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:34.499716 7472 consensus_queue.cc:260] T 8feefea4624e49309cef9ecb952bbe96 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:34.499883 7479 tablet_bootstrap.cc:492] T 3443c927080841c299ca239ac9eee2be P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:34.500051 7472 ts_tablet_manager.cc:1398] T 8feefea4624e49309cef9ecb952bbe96 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20240502 20:46:34.500197 7472 tablet_bootstrap.cc:492] T 5468fdb5d2a240acbda0546d54b46b74 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:34.500763 7479 tablet_bootstrap.cc:654] T 3443c927080841c299ca239ac9eee2be P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:34.501083 7472 tablet_bootstrap.cc:654] T 5468fdb5d2a240acbda0546d54b46b74 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:34.502123 7480 tablet_bootstrap.cc:492] T 5468fdb5d2a240acbda0546d54b46b74 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:34.502172 6616 consensus_queue.cc:1035] T 4afde5c1bf754a819f84bcc41212617d P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:34.502305 7480 ts_tablet_manager.cc:1367] T 5468fdb5d2a240acbda0546d54b46b74 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 20:46:34.502805 7480 raft_consensus.cc:348] T 5468fdb5d2a240acbda0546d54b46b74 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:34.502897 7480 raft_consensus.cc:374] T 5468fdb5d2a240acbda0546d54b46b74 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:34.502934 7480 raft_consensus.cc:729] T 5468fdb5d2a240acbda0546d54b46b74 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:34.503044 7480 consensus_queue.cc:260] T 5468fdb5d2a240acbda0546d54b46b74 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:34.503118 7473 tablet_bootstrap.cc:492] T 5468fdb5d2a240acbda0546d54b46b74 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:34.503196 7473 ts_tablet_manager.cc:1367] T 5468fdb5d2a240acbda0546d54b46b74 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 20:46:34.503680 7473 raft_consensus.cc:348] T 5468fdb5d2a240acbda0546d54b46b74 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:34.503834 7473 raft_consensus.cc:374] T 5468fdb5d2a240acbda0546d54b46b74 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:34.503911 7473 raft_consensus.cc:729] T 5468fdb5d2a240acbda0546d54b46b74 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:34.504114 7473 consensus_queue.cc:260] T 5468fdb5d2a240acbda0546d54b46b74 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:34.504508 7473 ts_tablet_manager.cc:1398] T 5468fdb5d2a240acbda0546d54b46b74 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:34.504745 7465 ts_tablet_manager.cc:1398] T 75ace61b6538462b9bd5c0820eeacca9 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20240502 20:46:34.506114 7480 ts_tablet_manager.cc:1398] T 5468fdb5d2a240acbda0546d54b46b74 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:34.507036 7480 tablet_bootstrap.cc:492] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:34.507364 31777 tablet_copy_service.cc:342] P e7ab6b9053664ee5aa71c7978e26fe6a: Request end of tablet copy session dbcdc6ac635b44b185c3eba63f66e9d6-75ace61b6538462b9bd5c0820eeacca9 received from {username='slave'} at 127.22.45.67:60059
I20240502 20:46:34.507436 31777 tablet_copy_service.cc:433] P e7ab6b9053664ee5aa71c7978e26fe6a: ending tablet copy session dbcdc6ac635b44b185c3eba63f66e9d6-75ace61b6538462b9bd5c0820eeacca9 on tablet 75ace61b6538462b9bd5c0820eeacca9 with peer dbcdc6ac635b44b185c3eba63f66e9d6
I20240502 20:46:34.507972 7480 tablet_bootstrap.cc:654] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:34.508612 7488 tablet_bootstrap.cc:492] T 4432691bad6b42619ef72c41e72d27a9 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:34.509267 7483 tablet_bootstrap.cc:492] T 8feefea4624e49309cef9ecb952bbe96 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:34.509558 7472 tablet_bootstrap.cc:492] T 5468fdb5d2a240acbda0546d54b46b74 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:34.509639 7472 ts_tablet_manager.cc:1367] T 5468fdb5d2a240acbda0546d54b46b74 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 20:46:34.510085 7472 raft_consensus.cc:348] T 5468fdb5d2a240acbda0546d54b46b74 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:34.510175 7472 raft_consensus.cc:374] T 5468fdb5d2a240acbda0546d54b46b74 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:34.510205 7472 raft_consensus.cc:729] T 5468fdb5d2a240acbda0546d54b46b74 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:34.510306 7472 consensus_queue.cc:260] T 5468fdb5d2a240acbda0546d54b46b74 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:34.510586 7472 ts_tablet_manager.cc:1398] T 5468fdb5d2a240acbda0546d54b46b74 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:34.510736 7472 tablet_bootstrap.cc:492] T 4432691bad6b42619ef72c41e72d27a9 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:34.510957 7488 tablet_bootstrap.cc:654] T 4432691bad6b42619ef72c41e72d27a9 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:34.511543 7472 tablet_bootstrap.cc:654] T 4432691bad6b42619ef72c41e72d27a9 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:34.514499 7472 tablet_bootstrap.cc:492] T 4432691bad6b42619ef72c41e72d27a9 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:34.514623 7472 ts_tablet_manager.cc:1367] T 4432691bad6b42619ef72c41e72d27a9 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:34.514793 7478 tablet_bootstrap.cc:654] T 8feefea4624e49309cef9ecb952bbe96 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:34.515071 7472 raft_consensus.cc:348] T 4432691bad6b42619ef72c41e72d27a9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:34.515216 7472 raft_consensus.cc:374] T 4432691bad6b42619ef72c41e72d27a9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:34.515295 7472 raft_consensus.cc:729] T 4432691bad6b42619ef72c41e72d27a9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:34.515455 7472 consensus_queue.cc:260] T 4432691bad6b42619ef72c41e72d27a9 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:34.515832 7472 ts_tablet_manager.cc:1398] T 4432691bad6b42619ef72c41e72d27a9 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:34.518996 7483 tablet_bootstrap.cc:654] T 8feefea4624e49309cef9ecb952bbe96 P 315915efcf2b413ca7f9d874338e1b2d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:34.521948 7479 tablet_bootstrap.cc:492] T 3443c927080841c299ca239ac9eee2be P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:34.522094 7479 ts_tablet_manager.cc:1367] T 3443c927080841c299ca239ac9eee2be P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20240502 20:46:34.522675 7479 raft_consensus.cc:348] T 3443c927080841c299ca239ac9eee2be P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:34.522927 7479 raft_consensus.cc:374] T 3443c927080841c299ca239ac9eee2be P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:34.522986 7479 raft_consensus.cc:729] T 3443c927080841c299ca239ac9eee2be P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:34.523152 7479 consensus_queue.cc:260] T 3443c927080841c299ca239ac9eee2be P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:34.524084 7479 ts_tablet_manager.cc:1398] T 3443c927080841c299ca239ac9eee2be P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:34.524849 7481 tablet_bootstrap.cc:492] T 3443c927080841c299ca239ac9eee2be P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:34.524950 7481 ts_tablet_manager.cc:1367] T 3443c927080841c299ca239ac9eee2be P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.036s user 0.002s sys 0.000s
I20240502 20:46:34.525125 7488 tablet_bootstrap.cc:492] T 4432691bad6b42619ef72c41e72d27a9 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:34.525233 7488 ts_tablet_manager.cc:1367] T 4432691bad6b42619ef72c41e72d27a9 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 20:46:34.525396 7481 raft_consensus.cc:348] T 3443c927080841c299ca239ac9eee2be P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:34.525497 7481 raft_consensus.cc:374] T 3443c927080841c299ca239ac9eee2be P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:34.525528 7481 raft_consensus.cc:729] T 3443c927080841c299ca239ac9eee2be P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:34.525697 7488 raft_consensus.cc:348] T 4432691bad6b42619ef72c41e72d27a9 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:34.526631 7488 raft_consensus.cc:374] T 4432691bad6b42619ef72c41e72d27a9 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:34.526661 7478 tablet_bootstrap.cc:492] T 8feefea4624e49309cef9ecb952bbe96 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:34.526687 7488 raft_consensus.cc:729] T 4432691bad6b42619ef72c41e72d27a9 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:34.526682 7481 consensus_queue.cc:260] T 3443c927080841c299ca239ac9eee2be P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:34.526742 7478 ts_tablet_manager.cc:1367] T 8feefea4624e49309cef9ecb952bbe96 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.032s user 0.001s sys 0.001s
I20240502 20:46:34.526813 7488 consensus_queue.cc:260] T 4432691bad6b42619ef72c41e72d27a9 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:34.526970 7481 ts_tablet_manager.cc:1398] T 3443c927080841c299ca239ac9eee2be P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:34.527160 7488 ts_tablet_manager.cc:1398] T 4432691bad6b42619ef72c41e72d27a9 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:34.527194 7483 tablet_bootstrap.cc:492] T 8feefea4624e49309cef9ecb952bbe96 P 315915efcf2b413ca7f9d874338e1b2d: No bootstrap required, opened a new log
I20240502 20:46:34.527271 7483 ts_tablet_manager.cc:1367] T 8feefea4624e49309cef9ecb952bbe96 P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240502 20:46:34.527706 7483 raft_consensus.cc:348] T 8feefea4624e49309cef9ecb952bbe96 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:34.528154 7478 raft_consensus.cc:348] T 8feefea4624e49309cef9ecb952bbe96 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:34.528266 7478 raft_consensus.cc:374] T 8feefea4624e49309cef9ecb952bbe96 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:34.528342 7483 raft_consensus.cc:374] T 8feefea4624e49309cef9ecb952bbe96 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:34.528393 7483 raft_consensus.cc:729] T 8feefea4624e49309cef9ecb952bbe96 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:34.528694 7483 consensus_queue.cc:260] T 8feefea4624e49309cef9ecb952bbe96 P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:34.528981 7478 raft_consensus.cc:729] T 8feefea4624e49309cef9ecb952bbe96 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:34.529287 7478 consensus_queue.cc:260] T 8feefea4624e49309cef9ecb952bbe96 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:34.531500 7478 ts_tablet_manager.cc:1398] T 8feefea4624e49309cef9ecb952bbe96 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 20:46:34.530450 7483 ts_tablet_manager.cc:1398] T 8feefea4624e49309cef9ecb952bbe96 P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240502 20:46:34.532369 7478 tablet_bootstrap.cc:492] T 3443c927080841c299ca239ac9eee2be P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:34.534134 7478 tablet_bootstrap.cc:654] T 3443c927080841c299ca239ac9eee2be P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:34.537112 7480 tablet_bootstrap.cc:492] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:34.537190 7480 ts_tablet_manager.cc:1367] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.000s
I20240502 20:46:34.537569 7480 raft_consensus.cc:348] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:34.537647 7480 raft_consensus.cc:374] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:34.537664 7480 raft_consensus.cc:729] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:34.537751 7480 consensus_queue.cc:260] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:34.538028 7480 ts_tablet_manager.cc:1398] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:34.541241 7478 tablet_bootstrap.cc:492] T 3443c927080841c299ca239ac9eee2be P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:34.541383 7478 ts_tablet_manager.cc:1367] T 3443c927080841c299ca239ac9eee2be P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20240502 20:46:34.541780 7478 raft_consensus.cc:348] T 3443c927080841c299ca239ac9eee2be P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:34.541867 7478 raft_consensus.cc:374] T 3443c927080841c299ca239ac9eee2be P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:34.541895 7478 raft_consensus.cc:729] T 3443c927080841c299ca239ac9eee2be P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:34.542022 7478 consensus_queue.cc:260] T 3443c927080841c299ca239ac9eee2be P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:34.543854 7478 ts_tablet_manager.cc:1398] T 3443c927080841c299ca239ac9eee2be P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240502 20:46:34.549335 7506 ts_tablet_manager.cc:902] T 60e8272de78a4c94b77408c4d1d5f60e P 315915efcf2b413ca7f9d874338e1b2d: Initiating tablet copy from peer dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:34.549825 7506 tablet_copy_client.cc:308] T 60e8272de78a4c94b77408c4d1d5f60e P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.67:44095
I20240502 20:46:34.551430 29465 tablet_copy_service.cc:140] P dbcdc6ac635b44b185c3eba63f66e9d6: Received BeginTabletCopySession request for tablet 60e8272de78a4c94b77408c4d1d5f60e from peer 315915efcf2b413ca7f9d874338e1b2d ({username='slave'} at 127.22.45.70:49775)
I20240502 20:46:34.551532 29465 tablet_copy_service.cc:161] P dbcdc6ac635b44b185c3eba63f66e9d6: Beginning new tablet copy session on tablet 60e8272de78a4c94b77408c4d1d5f60e from peer 315915efcf2b413ca7f9d874338e1b2d at {username='slave'} at 127.22.45.70:49775: session id = 315915efcf2b413ca7f9d874338e1b2d-60e8272de78a4c94b77408c4d1d5f60e
I20240502 20:46:34.552549 29465 tablet_copy_source_session.cc:205] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:34.553304 7506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60e8272de78a4c94b77408c4d1d5f60e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:34.555485 7506 tablet_copy_client.cc:783] T 60e8272de78a4c94b77408c4d1d5f60e P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:34.555491 3289 raft_consensus.cc:482] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:34.555568 3289 raft_consensus.cc:504] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:34.555624 7506 tablet_copy_client.cc:647] T 60e8272de78a4c94b77408c4d1d5f60e P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:34.555990 3289 leader_election.cc:288] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:34.556224 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4432691bad6b42619ef72c41e72d27a9" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:34.556339 29574 raft_consensus.cc:2454] T 4432691bad6b42619ef72c41e72d27a9 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:34.556808 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4432691bad6b42619ef72c41e72d27a9" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:34.556938 29444 raft_consensus.cc:2454] T 4432691bad6b42619ef72c41e72d27a9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:34.556973 31691 leader_election.cc:302] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:34.557590 3289 raft_consensus.cc:2781] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:34.557648 3289 raft_consensus.cc:482] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:34.557684 3289 raft_consensus.cc:3037] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:34.558761 3289 raft_consensus.cc:504] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:34.559096 3289 leader_election.cc:288] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:34.559470 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4432691bad6b42619ef72c41e72d27a9" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:34.559573 29574 raft_consensus.cc:3037] T 4432691bad6b42619ef72c41e72d27a9 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:34.559963 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4432691bad6b42619ef72c41e72d27a9" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:34.560053 29444 raft_consensus.cc:3037] T 4432691bad6b42619ef72c41e72d27a9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:34.560505 29574 raft_consensus.cc:2454] T 4432691bad6b42619ef72c41e72d27a9 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:34.560842 31691 leader_election.cc:302] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:34.560972 29444 raft_consensus.cc:2454] T 4432691bad6b42619ef72c41e72d27a9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:34.561125 7136 raft_consensus.cc:2781] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:34.561196 7136 raft_consensus.cc:686] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Becoming Leader. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Running, Role: LEADER
I20240502 20:46:34.561304 7136 consensus_queue.cc:237] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:34.565176 29041 catalog_manager.cc:5484] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: term changed from 0 to 1, leader changed from <none> to e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71). New cstate: current_term: 1 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:34.567373 7506 tablet_copy_client.cc:523] T 60e8272de78a4c94b77408c4d1d5f60e P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:34.568575 7506 tablet_bootstrap.cc:492] T 60e8272de78a4c94b77408c4d1d5f60e P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:34.573820 7506 tablet_bootstrap.cc:492] T 60e8272de78a4c94b77408c4d1d5f60e P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:34.574060 7506 tablet_bootstrap.cc:492] T 60e8272de78a4c94b77408c4d1d5f60e P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap complete.
I20240502 20:46:34.574206 7506 ts_tablet_manager.cc:1367] T 60e8272de78a4c94b77408c4d1d5f60e P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20240502 20:46:34.574637 7506 raft_consensus.cc:348] T 60e8272de78a4c94b77408c4d1d5f60e P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:34.574729 7506 raft_consensus.cc:729] T 60e8272de78a4c94b77408c4d1d5f60e P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:34.574839 7506 consensus_queue.cc:260] T 60e8272de78a4c94b77408c4d1d5f60e P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:34.575109 7506 ts_tablet_manager.cc:1398] T 60e8272de78a4c94b77408c4d1d5f60e P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:34.576869 29465 tablet_copy_service.cc:342] P dbcdc6ac635b44b185c3eba63f66e9d6: Request end of tablet copy session 315915efcf2b413ca7f9d874338e1b2d-60e8272de78a4c94b77408c4d1d5f60e received from {username='slave'} at 127.22.45.70:49775
I20240502 20:46:34.576941 29465 tablet_copy_service.cc:433] P dbcdc6ac635b44b185c3eba63f66e9d6: ending tablet copy session 315915efcf2b413ca7f9d874338e1b2d-60e8272de78a4c94b77408c4d1d5f60e on tablet 60e8272de78a4c94b77408c4d1d5f60e with peer 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:34.587471 7536 ts_tablet_manager.cc:902] T b28fe6bccf4f48a6a5456e6ead5cc920 P 2a9c1af8cf814030bde56b67ca5c2fbe: Initiating tablet copy from peer 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:34.588475 7536 tablet_copy_client.cc:308] T b28fe6bccf4f48a6a5456e6ead5cc920 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.65:43267
I20240502 20:46:34.588953 29202 tablet_copy_service.cc:140] P 7796ea3fba294d17a34386873969431d: Received BeginTabletCopySession request for tablet b28fe6bccf4f48a6a5456e6ead5cc920 from peer 2a9c1af8cf814030bde56b67ca5c2fbe ({username='slave'} at 127.22.45.69:49017)
I20240502 20:46:34.589044 29202 tablet_copy_service.cc:161] P 7796ea3fba294d17a34386873969431d: Beginning new tablet copy session on tablet b28fe6bccf4f48a6a5456e6ead5cc920 from peer 2a9c1af8cf814030bde56b67ca5c2fbe at {username='slave'} at 127.22.45.69:49017: session id = 2a9c1af8cf814030bde56b67ca5c2fbe-b28fe6bccf4f48a6a5456e6ead5cc920
I20240502 20:46:34.589807 29202 tablet_copy_source_session.cc:205] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:34.595507 7541 ts_tablet_manager.cc:902] T db9c5ce233da4b1a9ce70ae04aef7658 P e6b60803d0764d228c863e2da65e3691: Initiating tablet copy from peer dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:34.596066 7541 tablet_copy_client.cc:308] T db9c5ce233da4b1a9ce70ae04aef7658 P e6b60803d0764d228c863e2da65e3691: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.67:44095
I20240502 20:46:34.596484 29465 tablet_copy_service.cc:140] P dbcdc6ac635b44b185c3eba63f66e9d6: Received BeginTabletCopySession request for tablet db9c5ce233da4b1a9ce70ae04aef7658 from peer e6b60803d0764d228c863e2da65e3691 ({username='slave'} at 127.22.45.72:44455)
I20240502 20:46:34.596573 29465 tablet_copy_service.cc:161] P dbcdc6ac635b44b185c3eba63f66e9d6: Beginning new tablet copy session on tablet db9c5ce233da4b1a9ce70ae04aef7658 from peer e6b60803d0764d228c863e2da65e3691 at {username='slave'} at 127.22.45.72:44455: session id = e6b60803d0764d228c863e2da65e3691-db9c5ce233da4b1a9ce70ae04aef7658
I20240502 20:46:34.597491 29465 tablet_copy_source_session.cc:205] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:34.598239 7541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db9c5ce233da4b1a9ce70ae04aef7658. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:34.600415 7541 tablet_copy_client.cc:783] T db9c5ce233da4b1a9ce70ae04aef7658 P e6b60803d0764d228c863e2da65e3691: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:34.600569 7541 tablet_copy_client.cc:647] T db9c5ce233da4b1a9ce70ae04aef7658 P e6b60803d0764d228c863e2da65e3691: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:34.601519 7536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b28fe6bccf4f48a6a5456e6ead5cc920. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:34.603164 7541 tablet_copy_client.cc:523] T db9c5ce233da4b1a9ce70ae04aef7658 P e6b60803d0764d228c863e2da65e3691: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:34.604400 7536 tablet_copy_client.cc:783] T b28fe6bccf4f48a6a5456e6ead5cc920 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:34.613191 7541 tablet_bootstrap.cc:492] T db9c5ce233da4b1a9ce70ae04aef7658 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:34.613631 7536 tablet_copy_client.cc:647] T b28fe6bccf4f48a6a5456e6ead5cc920 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:34.621748 7536 tablet_copy_client.cc:523] T b28fe6bccf4f48a6a5456e6ead5cc920 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:34.625236 7536 tablet_bootstrap.cc:492] T b28fe6bccf4f48a6a5456e6ead5cc920 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:34.628499 7506 ts_tablet_manager.cc:902] T 8705260a889841bfa515683432dfe99d P 315915efcf2b413ca7f9d874338e1b2d: Initiating tablet copy from peer e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:34.628893 7506 tablet_copy_client.cc:308] T 8705260a889841bfa515683432dfe99d P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.71:36787
I20240502 20:46:34.629428 31777 tablet_copy_service.cc:140] P e7ab6b9053664ee5aa71c7978e26fe6a: Received BeginTabletCopySession request for tablet 8705260a889841bfa515683432dfe99d from peer 315915efcf2b413ca7f9d874338e1b2d ({username='slave'} at 127.22.45.70:34029)
I20240502 20:46:34.629528 31777 tablet_copy_service.cc:161] P e7ab6b9053664ee5aa71c7978e26fe6a: Beginning new tablet copy session on tablet 8705260a889841bfa515683432dfe99d from peer 315915efcf2b413ca7f9d874338e1b2d at {username='slave'} at 127.22.45.70:34029: session id = 315915efcf2b413ca7f9d874338e1b2d-8705260a889841bfa515683432dfe99d
I20240502 20:46:34.630431 31777 tablet_copy_source_session.cc:205] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:34.631912 7506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8705260a889841bfa515683432dfe99d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:34.634171 7506 tablet_copy_client.cc:783] T 8705260a889841bfa515683432dfe99d P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:34.634300 7506 tablet_copy_client.cc:647] T 8705260a889841bfa515683432dfe99d P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:34.638175 7506 tablet_copy_client.cc:523] T 8705260a889841bfa515683432dfe99d P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:34.639372 7506 tablet_bootstrap.cc:492] T 8705260a889841bfa515683432dfe99d P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:34.646872 7465 ts_tablet_manager.cc:902] T 43fc3dcc2dd54a11becb8f16ec500df1 P dbcdc6ac635b44b185c3eba63f66e9d6: Initiating tablet copy from peer 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:34.647264 7465 tablet_copy_client.cc:308] T 43fc3dcc2dd54a11becb8f16ec500df1 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.66:40945
I20240502 20:46:34.647711 7460 ts_tablet_manager.cc:902] T 358e4c761223404fbf0f467f38ce9b25 P 7796ea3fba294d17a34386873969431d: Initiating tablet copy from peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:34.647828 29335 tablet_copy_service.cc:140] P 09258ac095664fa99caf0707262cde13: Received BeginTabletCopySession request for tablet 43fc3dcc2dd54a11becb8f16ec500df1 from peer dbcdc6ac635b44b185c3eba63f66e9d6 ({username='slave'} at 127.22.45.67:43969)
I20240502 20:46:34.647918 29335 tablet_copy_service.cc:161] P 09258ac095664fa99caf0707262cde13: Beginning new tablet copy session on tablet 43fc3dcc2dd54a11becb8f16ec500df1 from peer dbcdc6ac635b44b185c3eba63f66e9d6 at {username='slave'} at 127.22.45.67:43969: session id = dbcdc6ac635b44b185c3eba63f66e9d6-43fc3dcc2dd54a11becb8f16ec500df1
I20240502 20:46:34.648140 7460 tablet_copy_client.cc:308] T 358e4c761223404fbf0f467f38ce9b25 P 7796ea3fba294d17a34386873969431d: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.70:46751
I20240502 20:46:34.648623 31646 tablet_copy_service.cc:140] P 315915efcf2b413ca7f9d874338e1b2d: Received BeginTabletCopySession request for tablet 358e4c761223404fbf0f467f38ce9b25 from peer 7796ea3fba294d17a34386873969431d ({username='slave'} at 127.22.45.65:33107)
I20240502 20:46:34.648705 31646 tablet_copy_service.cc:161] P 315915efcf2b413ca7f9d874338e1b2d: Beginning new tablet copy session on tablet 358e4c761223404fbf0f467f38ce9b25 from peer 7796ea3fba294d17a34386873969431d at {username='slave'} at 127.22.45.65:33107: session id = 7796ea3fba294d17a34386873969431d-358e4c761223404fbf0f467f38ce9b25
I20240502 20:46:34.648809 29335 tablet_copy_source_session.cc:205] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:34.649541 31646 tablet_copy_source_session.cc:205] T 358e4c761223404fbf0f467f38ce9b25 P 315915efcf2b413ca7f9d874338e1b2d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:34.649696 7465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43fc3dcc2dd54a11becb8f16ec500df1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:34.650425 7460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 358e4c761223404fbf0f467f38ce9b25. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:34.652245 7465 tablet_copy_client.cc:783] T 43fc3dcc2dd54a11becb8f16ec500df1 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:34.652388 7465 tablet_copy_client.cc:647] T 43fc3dcc2dd54a11becb8f16ec500df1 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:34.652555 7460 tablet_copy_client.cc:783] T 358e4c761223404fbf0f467f38ce9b25 P 7796ea3fba294d17a34386873969431d: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:34.654248 7460 tablet_copy_client.cc:647] T 358e4c761223404fbf0f467f38ce9b25 P 7796ea3fba294d17a34386873969431d: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:34.656407 7465 tablet_copy_client.cc:523] T 43fc3dcc2dd54a11becb8f16ec500df1 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:34.657441 7022 raft_consensus.cc:482] T 5468fdb5d2a240acbda0546d54b46b74 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:34.657545 7465 tablet_bootstrap.cc:492] T 43fc3dcc2dd54a11becb8f16ec500df1 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:34.657523 7022 raft_consensus.cc:504] T 5468fdb5d2a240acbda0546d54b46b74 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:34.657871 7022 leader_election.cc:288] T 5468fdb5d2a240acbda0546d54b46b74 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:34.658192 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5468fdb5d2a240acbda0546d54b46b74" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:34.658299 29444 raft_consensus.cc:2454] T 5468fdb5d2a240acbda0546d54b46b74 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:34.658591 31430 leader_election.cc:302] T 5468fdb5d2a240acbda0546d54b46b74 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:34.658751 7022 raft_consensus.cc:2781] T 5468fdb5d2a240acbda0546d54b46b74 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:34.658795 7022 raft_consensus.cc:482] T 5468fdb5d2a240acbda0546d54b46b74 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:34.658829 7022 raft_consensus.cc:3037] T 5468fdb5d2a240acbda0546d54b46b74 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:34.659838 7022 raft_consensus.cc:504] T 5468fdb5d2a240acbda0546d54b46b74 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:34.659902 3289 raft_consensus.cc:482] T 5468fdb5d2a240acbda0546d54b46b74 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:34.659988 3289 raft_consensus.cc:504] T 5468fdb5d2a240acbda0546d54b46b74 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:34.660158 7022 leader_election.cc:288] T 5468fdb5d2a240acbda0546d54b46b74 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Requested vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:34.660275 3289 leader_election.cc:288] T 5468fdb5d2a240acbda0546d54b46b74 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:34.660691 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5468fdb5d2a240acbda0546d54b46b74" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:34.660792 29444 raft_consensus.cc:3037] T 5468fdb5d2a240acbda0546d54b46b74 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:34.660866 7076 raft_consensus.cc:482] T 8feefea4624e49309cef9ecb952bbe96 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:34.660962 7076 raft_consensus.cc:504] T 8feefea4624e49309cef9ecb952bbe96 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:34.661379 7076 leader_election.cc:288] T 8feefea4624e49309cef9ecb952bbe96 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:34.661623 29444 raft_consensus.cc:2454] T 5468fdb5d2a240acbda0546d54b46b74 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:34.663022 7506 tablet_bootstrap.cc:492] T 8705260a889841bfa515683432dfe99d P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:34.663339 7506 tablet_bootstrap.cc:492] T 8705260a889841bfa515683432dfe99d P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap complete.
I20240502 20:46:34.663549 7506 ts_tablet_manager.cc:1367] T 8705260a889841bfa515683432dfe99d P 315915efcf2b413ca7f9d874338e1b2d: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.001s
I20240502 20:46:34.664147 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feefea4624e49309cef9ecb952bbe96" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d" is_pre_election: true
I20240502 20:46:34.664165 7506 raft_consensus.cc:348] T 8705260a889841bfa515683432dfe99d P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:34.664299 31625 raft_consensus.cc:2454] T 8feefea4624e49309cef9ecb952bbe96 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:34.664333 7506 raft_consensus.cc:729] T 8705260a889841bfa515683432dfe99d P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315915efcf2b413ca7f9d874338e1b2d, State: Initialized, Role: FOLLOWER
I20240502 20:46:34.664621 7506 consensus_queue.cc:260] T 8705260a889841bfa515683432dfe99d P 315915efcf2b413ca7f9d874338e1b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:34.664834 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5468fdb5d2a240acbda0546d54b46b74" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:34.664955 31755 raft_consensus.cc:2454] T 5468fdb5d2a240acbda0546d54b46b74 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:34.665109 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5468fdb5d2a240acbda0546d54b46b74" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:34.665223 31494 raft_consensus.cc:2379] T 5468fdb5d2a240acbda0546d54b46b74 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7ab6b9053664ee5aa71c7978e26fe6a in current term 1: Already voted for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in this term.
I20240502 20:46:34.665761 31430 leader_election.cc:302] T 5468fdb5d2a240acbda0546d54b46b74 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:34.666049 7506 ts_tablet_manager.cc:1398] T 8705260a889841bfa515683432dfe99d P 315915efcf2b413ca7f9d874338e1b2d: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:34.666035 31822 leader_election.cc:302] T 8feefea4624e49309cef9ecb952bbe96 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:34.666529 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5468fdb5d2a240acbda0546d54b46b74" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:34.666582 31777 tablet_copy_service.cc:342] P e7ab6b9053664ee5aa71c7978e26fe6a: Request end of tablet copy session 315915efcf2b413ca7f9d874338e1b2d-8705260a889841bfa515683432dfe99d received from {username='slave'} at 127.22.45.70:34029
I20240502 20:46:34.666646 31777 tablet_copy_service.cc:433] P e7ab6b9053664ee5aa71c7978e26fe6a: ending tablet copy session 315915efcf2b413ca7f9d874338e1b2d-8705260a889841bfa515683432dfe99d on tablet 8705260a889841bfa515683432dfe99d with peer 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:34.666651 29444 raft_consensus.cc:2379] T 5468fdb5d2a240acbda0546d54b46b74 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7ab6b9053664ee5aa71c7978e26fe6a in current term 1: Already voted for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in this term.
I20240502 20:46:34.666801 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feefea4624e49309cef9ecb952bbe96" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:34.666882 29446 raft_consensus.cc:2454] T 8feefea4624e49309cef9ecb952bbe96 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:34.667232 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5468fdb5d2a240acbda0546d54b46b74" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:34.667330 31755 raft_consensus.cc:3037] T 5468fdb5d2a240acbda0546d54b46b74 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:34.667698 7076 raft_consensus.cc:2781] T 8feefea4624e49309cef9ecb952bbe96 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:34.667748 7076 raft_consensus.cc:482] T 8feefea4624e49309cef9ecb952bbe96 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:34.667778 7076 raft_consensus.cc:3037] T 8feefea4624e49309cef9ecb952bbe96 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:34.668269 31755 raft_consensus.cc:2454] T 5468fdb5d2a240acbda0546d54b46b74 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:34.668423 7541 tablet_bootstrap.cc:492] T db9c5ce233da4b1a9ce70ae04aef7658 P e6b60803d0764d228c863e2da65e3691: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:34.669162 31692 leader_election.cc:302] T 5468fdb5d2a240acbda0546d54b46b74 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7ab6b9053664ee5aa71c7978e26fe6a; no voters: 2a9c1af8cf814030bde56b67ca5c2fbe, dbcdc6ac635b44b185c3eba63f66e9d6
I20240502 20:46:34.669322 3289 raft_consensus.cc:2726] T 5468fdb5d2a240acbda0546d54b46b74 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 20:46:34.669513 7022 raft_consensus.cc:2781] T 5468fdb5d2a240acbda0546d54b46b74 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:34.669592 7022 raft_consensus.cc:686] T 5468fdb5d2a240acbda0546d54b46b74 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Becoming Leader. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Running, Role: LEADER
I20240502 20:46:34.669701 7022 consensus_queue.cc:237] T 5468fdb5d2a240acbda0546d54b46b74 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:34.669945 7465 tablet_bootstrap.cc:492] T 43fc3dcc2dd54a11becb8f16ec500df1 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:34.670156 7465 tablet_bootstrap.cc:492] T 43fc3dcc2dd54a11becb8f16ec500df1 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap complete.
I20240502 20:46:34.670289 7465 ts_tablet_manager.cc:1367] T 43fc3dcc2dd54a11becb8f16ec500df1 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.004s
I20240502 20:46:34.670718 7465 raft_consensus.cc:348] T 43fc3dcc2dd54a11becb8f16ec500df1 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:34.670814 7465 raft_consensus.cc:729] T 43fc3dcc2dd54a11becb8f16ec500df1 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:34.670923 7465 consensus_queue.cc:260] T 43fc3dcc2dd54a11becb8f16ec500df1 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:34.671344 7465 ts_tablet_manager.cc:1398] T 43fc3dcc2dd54a11becb8f16ec500df1 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:34.673131 29335 tablet_copy_service.cc:342] P 09258ac095664fa99caf0707262cde13: Request end of tablet copy session dbcdc6ac635b44b185c3eba63f66e9d6-43fc3dcc2dd54a11becb8f16ec500df1 received from {username='slave'} at 127.22.45.67:43969
I20240502 20:46:34.673403 29335 tablet_copy_service.cc:433] P 09258ac095664fa99caf0707262cde13: ending tablet copy session dbcdc6ac635b44b185c3eba63f66e9d6-43fc3dcc2dd54a11becb8f16ec500df1 on tablet 43fc3dcc2dd54a11becb8f16ec500df1 with peer dbcdc6ac635b44b185c3eba63f66e9d6
I20240502 20:46:34.675097 29037 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:34.668686 7076 raft_consensus.cc:504] T 8feefea4624e49309cef9ecb952bbe96 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:34.676218 7076 leader_election.cc:288] T 8feefea4624e49309cef9ecb952bbe96 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:34.676663 7541 tablet_bootstrap.cc:492] T db9c5ce233da4b1a9ce70ae04aef7658 P e6b60803d0764d228c863e2da65e3691: Bootstrap complete.
I20240502 20:46:34.676852 7541 ts_tablet_manager.cc:1367] T db9c5ce233da4b1a9ce70ae04aef7658 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.064s user 0.021s sys 0.005s
I20240502 20:46:34.676875 31625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feefea4624e49309cef9ecb952bbe96" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315915efcf2b413ca7f9d874338e1b2d"
I20240502 20:46:34.676982 31625 raft_consensus.cc:3037] T 8feefea4624e49309cef9ecb952bbe96 P 315915efcf2b413ca7f9d874338e1b2d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:34.677934 31625 raft_consensus.cc:2454] T 8feefea4624e49309cef9ecb952bbe96 P 315915efcf2b413ca7f9d874338e1b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:34.678231 7460 tablet_copy_client.cc:523] T 358e4c761223404fbf0f467f38ce9b25 P 7796ea3fba294d17a34386873969431d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:34.674489 6850 raft_consensus.cc:482] T 3443c927080841c299ca239ac9eee2be P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:34.678536 6850 raft_consensus.cc:504] T 3443c927080841c299ca239ac9eee2be P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:34.678722 7541 raft_consensus.cc:348] T db9c5ce233da4b1a9ce70ae04aef7658 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:34.678659 29041 catalog_manager.cc:5484] T 5468fdb5d2a240acbda0546d54b46b74 P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69). New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:34.678807 7541 raft_consensus.cc:729] T db9c5ce233da4b1a9ce70ae04aef7658 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:34.678869 6850 leader_election.cc:288] T 3443c927080841c299ca239ac9eee2be P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:34.678944 7541 consensus_queue.cc:260] T db9c5ce233da4b1a9ce70ae04aef7658 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:34.679198 31822 leader_election.cc:302] T 8feefea4624e49309cef9ecb952bbe96 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315915efcf2b413ca7f9d874338e1b2d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:34.679625 7460 tablet_bootstrap.cc:492] T 358e4c761223404fbf0f467f38ce9b25 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:34.679873 6850 raft_consensus.cc:2781] T 8feefea4624e49309cef9ecb952bbe96 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:34.679951 6850 raft_consensus.cc:686] T 8feefea4624e49309cef9ecb952bbe96 P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Becoming Leader. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Running, Role: LEADER
I20240502 20:46:34.680063 6850 consensus_queue.cc:237] T 8feefea4624e49309cef9ecb952bbe96 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:34.680593 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3443c927080841c299ca239ac9eee2be" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:34.680716 29315 raft_consensus.cc:2454] T 3443c927080841c299ca239ac9eee2be P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:34.680857 7541 ts_tablet_manager.cc:1398] T db9c5ce233da4b1a9ce70ae04aef7658 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:34.681224 31823 leader_election.cc:302] T 3443c927080841c299ca239ac9eee2be P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:34.681383 7076 raft_consensus.cc:2781] T 3443c927080841c299ca239ac9eee2be P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:34.681438 7076 raft_consensus.cc:482] T 3443c927080841c299ca239ac9eee2be P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:34.681483 7076 raft_consensus.cc:3037] T 3443c927080841c299ca239ac9eee2be P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:34.682202 7076 raft_consensus.cc:504] T 3443c927080841c299ca239ac9eee2be P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:34.682885 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3443c927080841c299ca239ac9eee2be" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:34.683024 29182 raft_consensus.cc:3037] T 3443c927080841c299ca239ac9eee2be P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:34.683310 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3443c927080841c299ca239ac9eee2be" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:34.683398 29315 raft_consensus.cc:3037] T 3443c927080841c299ca239ac9eee2be P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:34.683924 29182 raft_consensus.cc:2454] T 3443c927080841c299ca239ac9eee2be P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:34.682885 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3443c927080841c299ca239ac9eee2be" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:34.684163 29175 raft_consensus.cc:2362] T 3443c927080841c299ca239ac9eee2be P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1. Re-sending same reply.
I20240502 20:46:34.684322 29315 raft_consensus.cc:2454] T 3443c927080841c299ca239ac9eee2be P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:34.685073 29574 raft_consensus.cc:1261] T 4432691bad6b42619ef72c41e72d27a9 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer e7ab6b9053664ee5aa71c7978e26fe6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:34.685652 3289 consensus_queue.cc:1035] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:34.686625 7460 tablet_bootstrap.cc:492] T 358e4c761223404fbf0f467f38ce9b25 P 7796ea3fba294d17a34386873969431d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:34.686758 29465 tablet_copy_service.cc:342] P dbcdc6ac635b44b185c3eba63f66e9d6: Request end of tablet copy session e6b60803d0764d228c863e2da65e3691-db9c5ce233da4b1a9ce70ae04aef7658 received from {username='slave'} at 127.22.45.72:44455
I20240502 20:46:34.686825 29465 tablet_copy_service.cc:433] P dbcdc6ac635b44b185c3eba63f66e9d6: ending tablet copy session e6b60803d0764d228c863e2da65e3691-db9c5ce233da4b1a9ce70ae04aef7658 on tablet db9c5ce233da4b1a9ce70ae04aef7658 with peer e6b60803d0764d228c863e2da65e3691
I20240502 20:46:34.686928 7460 tablet_bootstrap.cc:492] T 358e4c761223404fbf0f467f38ce9b25 P 7796ea3fba294d17a34386873969431d: Bootstrap complete.
I20240502 20:46:34.687070 7460 ts_tablet_manager.cc:1367] T 358e4c761223404fbf0f467f38ce9b25 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.001s
I20240502 20:46:34.682439 7076 leader_election.cc:288] T 3443c927080841c299ca239ac9eee2be P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:34.687258 29446 raft_consensus.cc:1261] T 4432691bad6b42619ef72c41e72d27a9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer e7ab6b9053664ee5aa71c7978e26fe6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:34.684898 31824 leader_election.cc:302] T 3443c927080841c299ca239ac9eee2be P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:34.687472 7460 raft_consensus.cc:348] T 358e4c761223404fbf0f467f38ce9b25 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:34.687569 7460 raft_consensus.cc:729] T 358e4c761223404fbf0f467f38ce9b25 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:34.687712 7460 consensus_queue.cc:260] T 358e4c761223404fbf0f467f38ce9b25 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:34.688325 7460 ts_tablet_manager.cc:1398] T 358e4c761223404fbf0f467f38ce9b25 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:34.688834 31646 tablet_copy_service.cc:342] P 315915efcf2b413ca7f9d874338e1b2d: Request end of tablet copy session 7796ea3fba294d17a34386873969431d-358e4c761223404fbf0f467f38ce9b25 received from {username='slave'} at 127.22.45.65:33107
I20240502 20:46:34.688908 31646 tablet_copy_service.cc:433] P 315915efcf2b413ca7f9d874338e1b2d: ending tablet copy session 7796ea3fba294d17a34386873969431d-358e4c761223404fbf0f467f38ce9b25 on tablet 358e4c761223404fbf0f467f38ce9b25 with peer 7796ea3fba294d17a34386873969431d
I20240502 20:46:34.689154 29041 catalog_manager.cc:5484] T 8feefea4624e49309cef9ecb952bbe96 P e6b60803d0764d228c863e2da65e3691 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6b60803d0764d228c863e2da65e3691 (127.22.45.72). New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:34.689445 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feefea4624e49309cef9ecb952bbe96" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:34.689553 29444 raft_consensus.cc:3037] T 8feefea4624e49309cef9ecb952bbe96 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:34.689705 29037 catalog_manager.cc:5576] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897) does not have the latest schema for tablet 4432691bad6b42619ef72c41e72d27a9 (table rebalancer_dynamic_table [id=e568822f039f4f97a3afbf915c8eaeca]). Expected version 1 got 0
I20240502 20:46:34.689857 3289 consensus_queue.cc:1035] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:34.690515 29444 raft_consensus.cc:2454] T 8feefea4624e49309cef9ecb952bbe96 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:34.692939 29035 catalog_manager.cc:5576] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095) does not have the latest schema for tablet 4432691bad6b42619ef72c41e72d27a9 (table rebalancer_dynamic_table [id=e568822f039f4f97a3afbf915c8eaeca]). Expected version 1 got 0
I20240502 20:46:34.692972 7536 tablet_bootstrap.cc:492] T b28fe6bccf4f48a6a5456e6ead5cc920 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:34.693202 7536 tablet_bootstrap.cc:492] T b28fe6bccf4f48a6a5456e6ead5cc920 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap complete.
I20240502 20:46:34.693414 7536 ts_tablet_manager.cc:1367] T b28fe6bccf4f48a6a5456e6ead5cc920 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.068s user 0.026s sys 0.000s
I20240502 20:46:34.693887 7536 raft_consensus.cc:348] T b28fe6bccf4f48a6a5456e6ead5cc920 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:34.694008 7076 raft_consensus.cc:2781] T 3443c927080841c299ca239ac9eee2be P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:34.694099 7076 raft_consensus.cc:686] T 3443c927080841c299ca239ac9eee2be P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Becoming Leader. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Running, Role: LEADER
I20240502 20:46:34.694056 7536 raft_consensus.cc:729] T b28fe6bccf4f48a6a5456e6ead5cc920 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:34.694219 7076 consensus_queue.cc:237] T 3443c927080841c299ca239ac9eee2be P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:34.694397 7536 consensus_queue.cc:260] T b28fe6bccf4f48a6a5456e6ead5cc920 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:34.694620 7170 tablet.cc:1722] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:34.694917 7170 tablet.cc:1620] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:34.695724 7536 ts_tablet_manager.cc:1398] T b28fe6bccf4f48a6a5456e6ead5cc920 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:34.696389 29202 tablet_copy_service.cc:342] P 7796ea3fba294d17a34386873969431d: Request end of tablet copy session 2a9c1af8cf814030bde56b67ca5c2fbe-b28fe6bccf4f48a6a5456e6ead5cc920 received from {username='slave'} at 127.22.45.69:49017
I20240502 20:46:34.696466 29202 tablet_copy_service.cc:433] P 7796ea3fba294d17a34386873969431d: ending tablet copy session 2a9c1af8cf814030bde56b67ca5c2fbe-b28fe6bccf4f48a6a5456e6ead5cc920 on tablet b28fe6bccf4f48a6a5456e6ead5cc920 with peer 2a9c1af8cf814030bde56b67ca5c2fbe
I20240502 20:46:34.697716 7569 tablet.cc:1722] T 4432691bad6b42619ef72c41e72d27a9 P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:34.697975 7569 tablet.cc:1620] T 4432691bad6b42619ef72c41e72d27a9 P dbcdc6ac635b44b185c3eba63f66e9d6: MemRowSet was empty: no flush needed.
I20240502 20:46:34.699009 29041 catalog_manager.cc:5484] T 3443c927080841c299ca239ac9eee2be P e6b60803d0764d228c863e2da65e3691 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6b60803d0764d228c863e2da65e3691 (127.22.45.72). New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:34.700562 7570 tablet.cc:1722] T 4432691bad6b42619ef72c41e72d27a9 P db17f26a73f3447b92aa9a947e3a7057: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:34.701529 29037 catalog_manager.cc:5576] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897) does not have the latest schema for tablet 4432691bad6b42619ef72c41e72d27a9 (table rebalancer_dynamic_table [id=e568822f039f4f97a3afbf915c8eaeca]). Expected version 1 got 0
I20240502 20:46:34.702245 7570 tablet.cc:1620] T 4432691bad6b42619ef72c41e72d27a9 P db17f26a73f3447b92aa9a947e3a7057: MemRowSet was empty: no flush needed.
I20240502 20:46:34.703190 29035 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=e568822f039f4f97a3afbf915c8eaeca] alter complete (version 1)
I20240502 20:46:34.703644 29037 catalog_manager.cc:5576] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095) does not have the latest schema for tablet 4432691bad6b42619ef72c41e72d27a9 (table rebalancer_dynamic_table [id=e568822f039f4f97a3afbf915c8eaeca]). Expected version 1 got 0
I20240502 20:46:34.721293 7460 ts_tablet_manager.cc:902] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 7796ea3fba294d17a34386873969431d: Initiating tablet copy from peer 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:34.721709 7460 tablet_copy_client.cc:308] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 7796ea3fba294d17a34386873969431d: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.69:43887
I20240502 20:46:34.722636 31602 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:34.722795 31474 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 20:46:34.722831 31515 tablet_copy_service.cc:140] P 2a9c1af8cf814030bde56b67ca5c2fbe: Received BeginTabletCopySession request for tablet 2227f1ae58a044ea955cffa8d8d8a3f2 from peer 7796ea3fba294d17a34386873969431d ({username='slave'} at 127.22.45.65:53409)
I20240502 20:46:34.722916 31515 tablet_copy_service.cc:161] P 2a9c1af8cf814030bde56b67ca5c2fbe: Beginning new tablet copy session on tablet 2227f1ae58a044ea955cffa8d8d8a3f2 from peer 7796ea3fba294d17a34386873969431d at {username='slave'} at 127.22.45.65:53409: session id = 7796ea3fba294d17a34386873969431d-2227f1ae58a044ea955cffa8d8d8a3f2
I20240502 20:46:34.723100 29295 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:34.723852 31515 tablet_copy_source_session.cc:205] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:34.725135 7460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2227f1ae58a044ea955cffa8d8d8a3f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:34.732664 7460 tablet_copy_client.cc:783] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 7796ea3fba294d17a34386873969431d: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:34.733338 7460 tablet_copy_client.cc:647] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 7796ea3fba294d17a34386873969431d: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:34.734145 29163 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 20:46:34.737584 31736 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:34.742404 29424 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 20:46:34.746804 29557 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:34.747426 7460 tablet_copy_client.cc:523] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 7796ea3fba294d17a34386873969431d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:34.749570 7460 tablet_bootstrap.cc:492] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:34.751339 7418 ts_tablet_manager.cc:902] T 2bec922ff95845d1a13c27526f689e15 P 09258ac095664fa99caf0707262cde13: Initiating tablet copy from peer db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:34.751803 7418 tablet_copy_client.cc:308] T 2bec922ff95845d1a13c27526f689e15 P 09258ac095664fa99caf0707262cde13: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.68:45897
I20240502 20:46:34.752310 29597 tablet_copy_service.cc:140] P db17f26a73f3447b92aa9a947e3a7057: Received BeginTabletCopySession request for tablet 2bec922ff95845d1a13c27526f689e15 from peer 09258ac095664fa99caf0707262cde13 ({username='slave'} at 127.22.45.66:53833)
I20240502 20:46:34.752403 29597 tablet_copy_service.cc:161] P db17f26a73f3447b92aa9a947e3a7057: Beginning new tablet copy session on tablet 2bec922ff95845d1a13c27526f689e15 from peer 09258ac095664fa99caf0707262cde13 at {username='slave'} at 127.22.45.66:53833: session id = 09258ac095664fa99caf0707262cde13-2bec922ff95845d1a13c27526f689e15
I20240502 20:46:34.753403 29597 tablet_copy_source_session.cc:205] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:34.754206 7460 tablet_bootstrap.cc:492] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 7796ea3fba294d17a34386873969431d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:34.754423 7418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bec922ff95845d1a13c27526f689e15. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:34.754452 7460 tablet_bootstrap.cc:492] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 7796ea3fba294d17a34386873969431d: Bootstrap complete.
I20240502 20:46:34.754616 7460 ts_tablet_manager.cc:1367] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240502 20:46:34.755010 7460 raft_consensus.cc:348] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:34.755128 7460 raft_consensus.cc:729] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:34.755263 7460 consensus_queue.cc:260] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:34.756661 7460 ts_tablet_manager.cc:1398] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:34.756717 7418 tablet_copy_client.cc:783] T 2bec922ff95845d1a13c27526f689e15 P 09258ac095664fa99caf0707262cde13: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:34.757089 7418 tablet_copy_client.cc:647] T 2bec922ff95845d1a13c27526f689e15 P 09258ac095664fa99caf0707262cde13: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:34.757627 31515 tablet_copy_service.cc:342] P 2a9c1af8cf814030bde56b67ca5c2fbe: Request end of tablet copy session 7796ea3fba294d17a34386873969431d-2227f1ae58a044ea955cffa8d8d8a3f2 received from {username='slave'} at 127.22.45.65:53409
I20240502 20:46:34.757701 31515 tablet_copy_service.cc:433] P 2a9c1af8cf814030bde56b67ca5c2fbe: ending tablet copy session 7796ea3fba294d17a34386873969431d-2227f1ae58a044ea955cffa8d8d8a3f2 on tablet 2227f1ae58a044ea955cffa8d8d8a3f2 with peer 7796ea3fba294d17a34386873969431d
I20240502 20:46:34.761250 31868 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:34.761673 7418 tablet_copy_client.cc:523] T 2bec922ff95845d1a13c27526f689e15 P 09258ac095664fa99caf0707262cde13: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:34.763026 7506 ts_tablet_manager.cc:902] T 9acc12c9f0464e07965c05285fd334f0 P 315915efcf2b413ca7f9d874338e1b2d: Initiating tablet copy from peer db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:34.763492 7506 tablet_copy_client.cc:308] T 9acc12c9f0464e07965c05285fd334f0 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.68:45897
I20240502 20:46:34.764127 29597 tablet_copy_service.cc:140] P db17f26a73f3447b92aa9a947e3a7057: Received BeginTabletCopySession request for tablet 9acc12c9f0464e07965c05285fd334f0 from peer 315915efcf2b413ca7f9d874338e1b2d ({username='slave'} at 127.22.45.70:48497)
I20240502 20:46:34.764214 29597 tablet_copy_service.cc:161] P db17f26a73f3447b92aa9a947e3a7057: Beginning new tablet copy session on tablet 9acc12c9f0464e07965c05285fd334f0 from peer 315915efcf2b413ca7f9d874338e1b2d at {username='slave'} at 127.22.45.70:48497: session id = 315915efcf2b413ca7f9d874338e1b2d-9acc12c9f0464e07965c05285fd334f0
I20240502 20:46:34.765014 29597 tablet_copy_source_session.cc:205] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:34.765292 7418 tablet_bootstrap.cc:492] T 2bec922ff95845d1a13c27526f689e15 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:34.765940 7506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9acc12c9f0464e07965c05285fd334f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:34.768436 7506 tablet_copy_client.cc:783] T 9acc12c9f0464e07965c05285fd334f0 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:34.768568 7506 tablet_copy_client.cc:647] T 9acc12c9f0464e07965c05285fd334f0 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:34.771432 7506 tablet_copy_client.cc:523] T 9acc12c9f0464e07965c05285fd334f0 P 315915efcf2b413ca7f9d874338e1b2d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:34.772492 7506 tablet_bootstrap.cc:492] T 9acc12c9f0464e07965c05285fd334f0 P 315915efcf2b413ca7f9d874338e1b2d: Bootstrap starting.
I20240502 20:46:34.774735 7418 tablet_bootstrap.cc:492] T 2bec922ff95845d1a13c27526f689e15 P 09258ac095664fa99caf0707262cde13: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:34.774971 7418 tablet_bootstrap.cc:492] T 2bec922ff95845d1a13c27526f689e15 P 09258ac095664fa99caf0707262cde13: Bootstrap complete.
I20240502 20:46:34.775121 7418 ts_tablet_manager.cc:1367] T 2bec922ff95845d1a13c27526f689e15 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.000s
I20240502 20:46:34.775547 7418 raft_consensus.cc:348] T 2bec922ff95845d1a13c27526f689e15 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:34.775655 7418 raft_consensus.cc:729] T 2bec922ff95845d1a13c27526f689e15 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:34.775772 7418 consensus_queue.cc:260] T 2bec922ff95845d1a13c27526f689e15 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:34.776204 7418 ts_tablet_manager.cc:1398] T 2bec922ff95845d1a13c27526f689e15 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 20:46:34.776659 29597 tablet_copy_service.cc:342] P db17f26a73f3447b92aa9a947e3a7057: Request end of tablet copy session 09258ac095664fa99caf0707262cde13-2bec922ff95845d1a13c27526f689e15 received from {username='slave'} at 127.22.45.66:53833
I20240502 20:46:34.776712 7599 ts_tablet_manager.cc:902] T 73c2e0de3fcf455ba1fecf701d23a05f P 315915efcf2b413ca7f9d874338e1b2d: Initiating tablet copy from peer e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:34.776736 29597 tablet_copy_service.cc:433] P db17f26a73f3447b92aa9a947e3a7057: ending tablet copy session 09258ac095664fa99caf0707262cde13-2bec922ff95845d1a13c27526f689e15 on tablet 2bec922ff95845d1a13c27526f689e15 with peer 09258ac095664fa99caf0707262cde13
F20240502 20:46:34.776880 7599 trace_metrics.cc:68] Check failed: g_intern_map->size() < 100 (100 vs. 100) Too many interned strings: (MaintenanceMgr.queue_time_us, MaintenanceMgr.queue_time_us) (MaintenanceMgr.run_cpu_time_us, MaintenanceMgr.run_cpu_time_us) (MaintenanceMgr.run_wall_time_us, MaintenanceMgr.run_wall_time_us) (apply.queue_time_us, apply.queue_time_us) (apply.run_cpu_time_us, apply.run_cpu_time_us) (apply.run_wall_time_us, apply.run_wall_time_us) (client-negotiator.queue_time_us, client-negotiator.queue_time_us) (client-negotiator.run_cpu_time_us, client-negotiator.run_cpu_time_us) (client-negotiator.run_wall_time_us, client-negotiator.run_wall_time_us) (compiler_manager_pool.queue_time_us, compiler_manager_pool.queue_time_us) (compiler_manager_pool.run_cpu_time_us, compiler_manager_pool.run_cpu_time_us) (compiler_manager_pool.run_wall_time_us, compiler_manager_pool.run_wall_time_us) (dirs.queue_time_us, dirs.queue_time_us) (dirs.run_cpu_time_us, dirs.run_cpu_time_us) (dirs.run_wall_time_us, dirs.run_wall_time_us) (dns-resolver.queue_time_us, dns-resolver.queue_time_us) (dns-resolver.run_cpu_time_us, dns-resolver.run_cpu_time_us) (dns-resolver.run_wall_time_us, dns-resolver.run_wall_time_us) (log-alloc.queue_time_us, log-alloc.queue_time_us) (log-alloc.run_cpu_time_us, log-alloc.run_cpu_time_us) (log-alloc.run_wall_time_us, log-alloc.run_wall_time_us) (prepare.queue_time_us, prepare.queue_time_us) (prepare.run_cpu_time_us, prepare.run_cpu_time_us) (prepare.run_wall_time_us, prepare.run_wall_time_us) (raft.queue_time_us, raft.queue_time_us) (raft.run_cpu_time_us, raft.run_cpu_time_us) (raft.run_wall_time_us, raft.run_wall_time_us) (repair_pool.queue_time_us, repair_pool.queue_time_us) (repair_pool.run_cpu_time_us, repair_pool.run_cpu_time_us) (repair_pool.run_wall_time_us, repair_pool.run_wall_time_us) (server-negotiator.queue_time_us, server-negotiator.queue_time_us) (server-negotiator.run_cpu_time_us, server-negotiator.run_cpu_time_us) (server-negotiator.run_wall_time_us, server-negotiator.run_wall_time_us) (tablet-copy.queue_time_us, tablet-copy.queue_time_us) (tablet-copy.run_cpu_time_us, tablet-copy.run_cpu_time_us) (tablet-copy.run_wall_time_us, tablet-copy.run_wall_time_us) (tablet-delete.queue_time_us, tablet-delete.queue_time_us) (tablet-delete.run_cpu_time_us, tablet-delete.run_cpu_time_us) (tablet-delete.run_wall_time_us, tablet-delete.run_wall_time_us) (tablet-download-pool-2c6dfe6a92284c5f96094eb16c757222.queue_time_us, tablet-download-pool-2c6dfe6a92284c5f96094eb16c757222.queue_time_us) (tablet-download-pool-2c6dfe6a92284c5f96094eb16c757222.run_cpu_time_us, tablet-download-pool-2c6dfe6a92284c5f96094eb16c757222.run_cpu_time_us) (tablet-download-pool-2c6dfe6a92284c5f96094eb16c757222.run_wall_time_us, tablet-download-pool-2c6dfe6a92284c5f96094eb16c757222.run_wall_time_us) (tablet-download-pool-2d72ef6e7e8d473baaff8f527c02332b.queue_time_us, tablet-download-pool-2d72ef6e7e8d473baaff8f527c02332b.queue_time_us) (tablet-download-pool-2d72ef6e7e8d473baaff8f527c02332b.run_cpu_time_us, tablet-download-pool-2d72ef6e7e8d473baaff8f527c02332b.run_cpu_time_us) (tablet-download-pool-2d72ef6e7e8d473baaff8f527c02332b.run_wall_time_us, tablet-download-pool-2d72ef6e7e8d473baaff8f527c02332b.run_wall_time_us) (tablet-download-pool-3c3bdc0b9e934ae2b81317a371817563.queue_time_us, tablet-download-pool-3c3bdc0b9e934ae2b81317a371817563.queue_time_us) (tablet-download-pool-3c3bdc0b9e934ae2b81317a371817563.run_cpu_time_us, tablet-download-pool-3c3bdc0b9e934ae2b81317a371817563.run_cpu_time_us) (tablet-download-pool-3c3bdc0b9e934ae2b81317a371817563.run_wall_time_us, tablet-download-pool-3c3bdc0b9e934ae2b81317a371817563.run_wall_time_us) (tablet-download-pool-4a91eafc5ecf45b09f60d712884b7c79.queue_time_us, tablet-download-pool-4a91eafc5ecf45b09f60d712884b7c79.queue_time_us) (tablet-download-pool-4a91eafc5ecf45b09f60d712884b7c79.run_cpu_time_us, tablet-download-pool-4a91eafc5ecf45b09f60d712884b7c79.run_cpu_time_us) (tablet-download-pool-4a91eafc5ecf45b09f60d712884b7c79.run_wall_time_us, tablet-download-pool-4a91eafc5ecf45b09f60d712884b7c79.run_wall_time_us) (tablet-download-pool-4bfb24d76b434cd5b138f868b1f29508.queue_time_us, tablet-download-pool-4bfb24d76b434cd5b138f868b1f29508.queue_time_us) (tablet-download-pool-4bfb24d76b434cd5b138f868b1f29508.run_cpu_time_us, tablet-download-pool-4bfb24d76b434cd5b138f868b1f29508.run_cpu_time_us) (tablet-download-pool-4bfb24d76b434cd5b138f868b1f29508.run_wall_time_us, tablet-download-pool-4bfb24d76b434cd5b138f868b1f29508.run_wall_time_us) (tablet-download-pool-60e8272de78a4c94b77408c4d1d5f60e.queue_time_us, tablet-download-pool-60e8272de78a4c94b77408c4d1d5f60e.queue_time_us) (tablet-download-pool-60e8272de78a4c94b77408c4d1d5f60e.run_cpu_time_us, tablet-download-pool-60e8272de78a4c94b77408c4d1d5f60e.run_cpu_time_us) (tablet-download-pool-60e8272de78a4c94b77408c4d1d5f60e.run_wall_time_us, tablet-download-pool-60e8272de78a4c94b77408c4d1d5f60e.run_wall_time_us) (tablet-download-pool-73c2e0de3fcf455ba1fecf701d23a05f.queue_time_us, tablet-download-pool-73c2e0de3fcf455ba1fecf701d23a05f.queue_time_us) (tablet-download-pool-846f1d6e39e642cfaabf96ad0c52366d.queue_time_us, tablet-download-pool-846f1d6e39e642cfaabf96ad0c52366d.queue_time_us) (tablet-download-pool-846f1d6e39e642cfaabf96ad0c52366d.run_cpu_time_us, tablet-download-pool-846f1d6e39e642cfaabf96ad0c52366d.run_cpu_time_us) (tablet-download-pool-846f1d6e39e642cfaabf96ad0c52366d.run_wall_time_us, tablet-download-pool-846f1d6e39e642cfaabf96ad0c52366d.run_wall_time_us) (tablet-download-pool-8705260a889841bfa515683432dfe99d.queue_time_us, tablet-download-pool-8705260a889841bfa515683432dfe99d.queue_time_us) (tablet-download-pool-8705260a889841bfa515683432dfe99d.run_cpu_time_us, tablet-download-pool-8705260a889841bfa515683432dfe99d.run_cpu_time_us) (tablet-download-pool-8705260a889841bfa515683432dfe99d.run_wall_time_us, tablet-download-pool-8705260a889841bfa515683432dfe99d.run_wall_time_us) (tablet-download-pool-9acc12c9f0464e07965c05285fd334f0.queue_time_us, tablet-download-pool-9acc12c9f0464e07965c05285fd334f0.queue_time_us) (tablet-download-pool-9acc12c9f0464e07965c05285fd334f0.run_cpu_time_us, tablet-download-pool-9acc12c9f0464e07965c05285fd334f0.run_cpu_time_us) (tablet-download-pool-9acc12c9f0464e07965c05285fd334f0.run_wall_time_us, tablet-download-pool-9acc12c9f0464e07965c05285fd334f0.run_wall_time_us) (tablet-download-pool-b0e8b1909df34fce92f94d6af5f5c3b6.queue_time_us, tablet-download-pool-b0e8b1909df34fce92f94d6af5f5c3b6.queue_time_us) (tablet-download-pool-b0e8b1909df34fce92f94d6af5f5c3b6.run_cpu_time_us, tablet-download-pool-b0e8b1909df34fce92f94d6af5f5c3b6.run_cpu_time_us) (tablet-download-pool-b0e8b1909df34fce92f94d6af5f5c3b6.run_wall_time_us, tablet-download-pool-b0e8b1909df34fce92f94d6af5f5c3b6.run_wall_time_us) (tablet-download-pool-b28fe6bccf4f48a6a5456e6ead5cc920.queue_time_us, tablet-download-pool-b28fe6bccf4f48a6a5456e6ead5cc920.queue_time_us) (tablet-download-pool-b28fe6bccf4f48a6a5456e6ead5cc920.run_cpu_time_us, tablet-download-pool-b28fe6bccf4f48a6a5456e6ead5cc920.run_cpu_time_us) (tablet-download-pool-b28fe6bccf4f48a6a5456e6ead5cc920.run_wall_time_us, tablet-download-pool-b28fe6bccf4f48a6a5456e6ead5cc920.run_wall_time_us) (tablet-download-pool-d4a2c7b197f948a0b236d5e05e2749b9.queue_time_us, tablet-download-pool-d4a2c7b197f948a0b236d5e05e2749b9.queue_time_us) (tablet-download-pool-d4a2c7b197f948a0b236d5e05e2749b9.run_cpu_time_us, tablet-download-pool-d4a2c7b197f948a0b236d5e05e2749b9.run_cpu_time_us) (tablet-download-pool-d4a2c7b197f948a0b236d5e05e2749b9.run_wall_time_us, tablet-download-pool-d4a2c7b197f948a0b236d5e05e2749b9.run_wall_time_us) (tablet-download-pool-db9c5ce233da4b1a9ce70ae04aef7658.queue_time_us, tablet-download-pool-db9c5ce233da4b1a9ce70ae04aef7658.queue_time_us) (tablet-download-pool-db9c5ce233da4b1a9ce70ae04aef7658.run_cpu_time_us, tablet-download-pool-db9c5ce233da4b1a9ce70ae04aef7658.run_cpu_time_us) (tablet-download-pool-db9c5ce233da4b1a9ce70ae04aef7658.run_wall_time_us, tablet-download-pool-db9c5ce233da4b1a9ce70ae04aef7658.run_wall_time_us) (tablet-download-pool-fbd2bd8e142741ba864e2f99610b39d6.queue_time_us, tablet-download-pool-fbd2bd8e142741ba864e2f99610b39d6.queue_time_us) (tablet-download-pool-fbd2bd8e142741ba864e2f99610b39d6.run_cpu_time_us, tablet-download-pool-fbd2bd8e142741ba864e2f99610b39d6.run_cpu_time_us) (tablet-download-pool-fbd2bd8e142741ba864e2f99610b39d6.run_wall_time_us, tablet-download-pool-fbd2bd8e142741ba864e2f99610b39d6.run_wall_time_us) (tablet-open.queue_time_us, tablet-open.queue_time_us) (tablet-open.run_cpu_time_us, tablet-open.run_cpu_time_us) (tablet-open.run_wall_time_us, tablet-open.run_wall_time_us) (txn-commit.queue_time_us, txn-commit.queue_time_us) (txn-commit.run_cpu_time_us, txn-commit.run_cpu_time_us) (txn-commit.run_wall_time_us, txn-commit.run_wall_time_us) (txn-participant-registration.queue_time_us, txn-participant-registration.queue_time_us) (txn-participant-registration.run_cpu_time_us, txn-participant-registration.run_cpu_time_us) (txn-participant-registration.run_wall_time_us, txn-participant-registration.run_wall_time_us) (txn-status-manager.queue_time_us, txn-status-manager.queue_time_us) (txn-status-manager.run_cpu_time_us, txn-status-manager.run_cpu_time_us) (txn-status-manager.run_wall_time_us, txn-status-manager.run_wall_time_us) (txn-status-tablet-reload.queue_time_us, txn-status-tablet-reload.queue_time_us) (txn-status-tablet-reload.run_cpu_time_us, txn-status-tablet-reload.run_cpu_time_us) (txn-status-tablet-reload.run_wall_time_us, txn-status-tablet-reload.run_wall_time_us) (wal-append.queue_time_us, wal-append.queue_time_us) (wal-append.run_cpu_time_us, wal-append.run_cpu_time_us) (wal-append.run_wall_time_us, wal-append.run_wall_time_us)
*** Check failure stack trace: ***
@ 0x7f5527d20dcd google::LogMessage::Fail() at ??:0
@ 0x7f5527d24b93 google::LogMessage::SendToLog() at ??:0
@ 0x7f5527d207cc google::LogMessage::Flush() at ??:0
@ 0x7f5527d21f59 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f552918a044 kudu::TraceMetrics::InternName() at ??:0
@ 0x7f552916ef98 kudu::ThreadPool::ThreadPool() at ??:0
@ 0x7f552916cefc kudu::ThreadPoolBuilder::Build() at ??:0
@ 0x7f552bd1f77a kudu::tserver::TabletCopyClient::TabletCopyClient() at ??:0
I20240502 20:46:34.808043 29037 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
@ 0x7f552bd28a45 kudu::tserver::RemoteTabletCopyClient::RemoteTabletCopyClient() at ??:0
I20240502 20:46:34.815698 7170 tablet.cc:1722] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
@ 0x7f552bda976f kudu::tserver::TSTabletManager::RunTabletCopy() at ??:0
I20240502 20:46:34.816031 7170 tablet.cc:1620] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:34.817467 7570 tablet.cc:1722] T 4432691bad6b42619ef72c41e72d27a9 P db17f26a73f3447b92aa9a947e3a7057: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:34.817737 7570 tablet.cc:1620] T 4432691bad6b42619ef72c41e72d27a9 P db17f26a73f3447b92aa9a947e3a7057: MemRowSet was empty: no flush needed.
@ 0x7f552bdbd4f6 kudu::tserver::TabletCopyRunnable::Run() at ??:0
I20240502 20:46:34.821717 7607 ts_tablet_manager.cc:902] T d5ca50fc288f4957a2d6f04c23bd1491 P db17f26a73f3447b92aa9a947e3a7057: Initiating tablet copy from peer 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:34.822163 7607 tablet_copy_client.cc:308] T d5ca50fc288f4957a2d6f04c23bd1491 P db17f26a73f3447b92aa9a947e3a7057: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.66:40945
I20240502 20:46:34.822593 29335 tablet_copy_service.cc:140] P 09258ac095664fa99caf0707262cde13: Received BeginTabletCopySession request for tablet d5ca50fc288f4957a2d6f04c23bd1491 from peer db17f26a73f3447b92aa9a947e3a7057 ({username='slave'} at 127.22.45.68:56315)
I20240502 20:46:34.822688 29335 tablet_copy_service.cc:161] P 09258ac095664fa99caf0707262cde13: Beginning new tablet copy session on tablet d5ca50fc288f4957a2d6f04c23bd1491 from peer db17f26a73f3447b92aa9a947e3a7057 at {username='slave'} at 127.22.45.68:56315: session id = db17f26a73f3447b92aa9a947e3a7057-d5ca50fc288f4957a2d6f04c23bd1491
I20240502 20:46:34.822801 7569 tablet.cc:1722] T 4432691bad6b42619ef72c41e72d27a9 P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:34.823067 7569 tablet.cc:1620] T 4432691bad6b42619ef72c41e72d27a9 P dbcdc6ac635b44b185c3eba63f66e9d6: MemRowSet was empty: no flush needed.
I20240502 20:46:34.823626 29335 tablet_copy_source_session.cc:205] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:34.824411 29037 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=e568822f039f4f97a3afbf915c8eaeca] alter complete (version 2)
I20240502 20:46:34.825011 7607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5ca50fc288f4957a2d6f04c23bd1491. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:34.826175 7536 ts_tablet_manager.cc:902] T c8dc196b2d4e48b195524bc79c07b9eb P 2a9c1af8cf814030bde56b67ca5c2fbe: Initiating tablet copy from peer 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:34.826728 7536 tablet_copy_client.cc:308] T c8dc196b2d4e48b195524bc79c07b9eb P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.65:43267
I20240502 20:46:34.827353 29202 tablet_copy_service.cc:140] P 7796ea3fba294d17a34386873969431d: Received BeginTabletCopySession request for tablet c8dc196b2d4e48b195524bc79c07b9eb from peer 2a9c1af8cf814030bde56b67ca5c2fbe ({username='slave'} at 127.22.45.69:49017)
I20240502 20:46:34.827445 29202 tablet_copy_service.cc:161] P 7796ea3fba294d17a34386873969431d: Beginning new tablet copy session on tablet c8dc196b2d4e48b195524bc79c07b9eb from peer 2a9c1af8cf814030bde56b67ca5c2fbe at {username='slave'} at 127.22.45.69:49017: session id = 2a9c1af8cf814030bde56b67ca5c2fbe-c8dc196b2d4e48b195524bc79c07b9eb
I20240502 20:46:34.827697 7607 tablet_copy_client.cc:783] T d5ca50fc288f4957a2d6f04c23bd1491 P db17f26a73f3447b92aa9a947e3a7057: tablet copy: Starting download of 0 data blocks...
@ 0x7f552bda8226 _ZZN4kudu7tserver15TSTabletManager15StartTabletCopyEPKNS_9consensus24StartTabletCopyRequestPBESt8functionIFvRKNS_6StatusENS0_24TabletServerErrorPB_CodeEEEENKUlvE_clEv at ??:0
I20240502 20:46:34.827841 7607 tablet_copy_client.cc:647] T d5ca50fc288f4957a2d6f04c23bd1491 P db17f26a73f3447b92aa9a947e3a7057: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:34.828166 29202 tablet_copy_source_session.cc:205] T c8dc196b2d4e48b195524bc79c07b9eb P 7796ea3fba294d17a34386873969431d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:34.829077 7607 tablet_copy_client.cc:523] T d5ca50fc288f4957a2d6f04c23bd1491 P db17f26a73f3447b92aa9a947e3a7057: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:34.829679 7536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8dc196b2d4e48b195524bc79c07b9eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:34.830793 7607 tablet_bootstrap.cc:492] T d5ca50fc288f4957a2d6f04c23bd1491 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:34.831807 7536 tablet_copy_client.cc:783] T c8dc196b2d4e48b195524bc79c07b9eb P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:34.831947 7536 tablet_copy_client.cc:647] T c8dc196b2d4e48b195524bc79c07b9eb P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:34.833480 7536 tablet_copy_client.cc:523] T c8dc196b2d4e48b195524bc79c07b9eb P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet copy: Tablet Copy complete. Replacing tablet superblock.
@ 0x7f552bdb7c0a _ZNSt17_Function_handlerIFvvEZN4kudu7tserver15TSTabletManager15StartTabletCopyEPKNS1_9consensus24StartTabletCopyRequestPBESt8functionIFvRKNS1_6StatusENS2_24TabletServerErrorPB_CodeEEEEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
I20240502 20:46:34.834513 7536 tablet_bootstrap.cc:492] T c8dc196b2d4e48b195524bc79c07b9eb P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
@ 0x555f289e375c std::function<>::operator()() at ??:0
@ 0x7f5529171e06 kudu::ThreadPool::DispatchThread() at ??:0
I20240502 20:46:34.837975 7418 ts_tablet_manager.cc:902] T 7db3d66689ef4fb2869e94448515ad12 P 09258ac095664fa99caf0707262cde13: Initiating tablet copy from peer e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:34.838389 7418 tablet_copy_client.cc:308] T 7db3d66689ef4fb2869e94448515ad12 P 09258ac095664fa99caf0707262cde13: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.71:36787
I20240502 20:46:34.838845 31777 tablet_copy_service.cc:140] P e7ab6b9053664ee5aa71c7978e26fe6a: Received BeginTabletCopySession request for tablet 7db3d66689ef4fb2869e94448515ad12 from peer 09258ac095664fa99caf0707262cde13 ({username='slave'} at 127.22.45.66:51015)
I20240502 20:46:34.838953 31777 tablet_copy_service.cc:161] P e7ab6b9053664ee5aa71c7978e26fe6a: Beginning new tablet copy session on tablet 7db3d66689ef4fb2869e94448515ad12 from peer 09258ac095664fa99caf0707262cde13 at {username='slave'} at 127.22.45.66:51015: session id = 09258ac095664fa99caf0707262cde13-7db3d66689ef4fb2869e94448515ad12
@ 0x7f55291726ef _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
I20240502 20:46:34.839782 31777 tablet_copy_source_session.cc:205] T 7db3d66689ef4fb2869e94448515ad12 P e7ab6b9053664ee5aa71c7978e26fe6a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:34.841019 7607 tablet_bootstrap.cc:492] T d5ca50fc288f4957a2d6f04c23bd1491 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:34.841248 7607 tablet_bootstrap.cc:492] T d5ca50fc288f4957a2d6f04c23bd1491 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap complete.
I20240502 20:46:34.841385 7607 ts_tablet_manager.cc:1367] T d5ca50fc288f4957a2d6f04c23bd1491 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
@ 0x7f552917403b _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
I20240502 20:46:34.841835 7607 raft_consensus.cc:348] T d5ca50fc288f4957a2d6f04c23bd1491 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:34.841939 7607 raft_consensus.cc:729] T d5ca50fc288f4957a2d6f04c23bd1491 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:34.842087 7607 consensus_queue.cc:260] T d5ca50fc288f4957a2d6f04c23bd1491 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
@ 0x555f289e375c std::function<>::operator()() at ??:0
I20240502 20:46:34.843627 7607 ts_tablet_manager.cc:1398] T d5ca50fc288f4957a2d6f04c23bd1491 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:34.844044 29335 tablet_copy_service.cc:342] P 09258ac095664fa99caf0707262cde13: Request end of tablet copy session db17f26a73f3447b92aa9a947e3a7057-d5ca50fc288f4957a2d6f04c23bd1491 received from {username='slave'} at 127.22.45.68:56315
I20240502 20:46:34.844110 29335 tablet_copy_service.cc:433] P 09258ac095664fa99caf0707262cde13: ending tablet copy session db17f26a73f3447b92aa9a947e3a7057-d5ca50fc288f4957a2d6f04c23bd1491 on tablet d5ca50fc288f4957a2d6f04c23bd1491 with peer db17f26a73f3447b92aa9a947e3a7057
I20240502 20:46:34.847190 7418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7db3d66689ef4fb2869e94448515ad12. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f5529161807 kudu::Thread::SuperviseThread() at ??:0
I20240502 20:46:34.849328 7418 tablet_copy_client.cc:783] T 7db3d66689ef4fb2869e94448515ad12 P 09258ac095664fa99caf0707262cde13: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:34.849472 7418 tablet_copy_client.cc:647] T 7db3d66689ef4fb2869e94448515ad12 P 09258ac095664fa99caf0707262cde13: tablet copy: Starting download of 1 WAL segments...
@ 0x7f552999f6db start_thread at ??:0
I20240502 20:46:34.850351 7536 tablet_bootstrap.cc:492] T c8dc196b2d4e48b195524bc79c07b9eb P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:34.850618 7536 tablet_bootstrap.cc:492] T c8dc196b2d4e48b195524bc79c07b9eb P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap complete.
I20240502 20:46:34.850764 7536 ts_tablet_manager.cc:1367] T c8dc196b2d4e48b195524bc79c07b9eb P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.001s
I20240502 20:46:34.851238 7536 raft_consensus.cc:348] T c8dc196b2d4e48b195524bc79c07b9eb P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:34.851344 7536 raft_consensus.cc:729] T c8dc196b2d4e48b195524bc79c07b9eb P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:34.851507 7536 consensus_queue.cc:260] T c8dc196b2d4e48b195524bc79c07b9eb P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
@ 0x7f5526ed771f clone at ??:0
I20240502 20:46:34.852923 7536 ts_tablet_manager.cc:1398] T c8dc196b2d4e48b195524bc79c07b9eb P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:34.853458 29202 tablet_copy_service.cc:342] P 7796ea3fba294d17a34386873969431d: Request end of tablet copy session 2a9c1af8cf814030bde56b67ca5c2fbe-c8dc196b2d4e48b195524bc79c07b9eb received from {username='slave'} at 127.22.45.69:49017
I20240502 20:46:34.853526 29202 tablet_copy_service.cc:433] P 7796ea3fba294d17a34386873969431d: ending tablet copy session 2a9c1af8cf814030bde56b67ca5c2fbe-c8dc196b2d4e48b195524bc79c07b9eb on tablet c8dc196b2d4e48b195524bc79c07b9eb with peer 2a9c1af8cf814030bde56b67ca5c2fbe
I20240502 20:46:34.855803 7418 tablet_copy_client.cc:523] T 7db3d66689ef4fb2869e94448515ad12 P 09258ac095664fa99caf0707262cde13: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:34.856993 7418 tablet_bootstrap.cc:492] T 7db3d66689ef4fb2869e94448515ad12 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:34.860621 7418 tablet_bootstrap.cc:492] T 7db3d66689ef4fb2869e94448515ad12 P 09258ac095664fa99caf0707262cde13: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:34.860850 7418 tablet_bootstrap.cc:492] T 7db3d66689ef4fb2869e94448515ad12 P 09258ac095664fa99caf0707262cde13: Bootstrap complete.
I20240502 20:46:34.860986 7418 ts_tablet_manager.cc:1367] T 7db3d66689ef4fb2869e94448515ad12 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:34.861397 7418 raft_consensus.cc:348] T 7db3d66689ef4fb2869e94448515ad12 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:34.861511 7418 raft_consensus.cc:729] T 7db3d66689ef4fb2869e94448515ad12 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:34.861635 7418 consensus_queue.cc:260] T 7db3d66689ef4fb2869e94448515ad12 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:34.862632 7418 ts_tablet_manager.cc:1398] T 7db3d66689ef4fb2869e94448515ad12 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:34.863852 31777 tablet_copy_service.cc:342] P e7ab6b9053664ee5aa71c7978e26fe6a: Request end of tablet copy session 09258ac095664fa99caf0707262cde13-7db3d66689ef4fb2869e94448515ad12 received from {username='slave'} at 127.22.45.66:51015
I20240502 20:46:34.863921 31777 tablet_copy_service.cc:433] P e7ab6b9053664ee5aa71c7978e26fe6a: ending tablet copy session 09258ac095664fa99caf0707262cde13-7db3d66689ef4fb2869e94448515ad12 on tablet 7db3d66689ef4fb2869e94448515ad12 with peer 09258ac095664fa99caf0707262cde13
I20240502 20:46:34.914745 7418 ts_tablet_manager.cc:902] T b7c80c3b3a5d4505b7dbbac6bab1de59 P 09258ac095664fa99caf0707262cde13: Initiating tablet copy from peer dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:34.915237 7418 tablet_copy_client.cc:308] T b7c80c3b3a5d4505b7dbbac6bab1de59 P 09258ac095664fa99caf0707262cde13: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.67:44095
I20240502 20:46:34.915701 29465 tablet_copy_service.cc:140] P dbcdc6ac635b44b185c3eba63f66e9d6: Received BeginTabletCopySession request for tablet b7c80c3b3a5d4505b7dbbac6bab1de59 from peer 09258ac095664fa99caf0707262cde13 ({username='slave'} at 127.22.45.66:57919)
I20240502 20:46:34.915788 29465 tablet_copy_service.cc:161] P dbcdc6ac635b44b185c3eba63f66e9d6: Beginning new tablet copy session on tablet b7c80c3b3a5d4505b7dbbac6bab1de59 from peer 09258ac095664fa99caf0707262cde13 at {username='slave'} at 127.22.45.66:57919: session id = 09258ac095664fa99caf0707262cde13-b7c80c3b3a5d4505b7dbbac6bab1de59
I20240502 20:46:34.916476 29465 tablet_copy_source_session.cc:205] T b7c80c3b3a5d4505b7dbbac6bab1de59 P dbcdc6ac635b44b185c3eba63f66e9d6: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:34.917311 7418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7c80c3b3a5d4505b7dbbac6bab1de59. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:34.924121 7418 tablet_copy_client.cc:783] T b7c80c3b3a5d4505b7dbbac6bab1de59 P 09258ac095664fa99caf0707262cde13: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:34.924363 7418 tablet_copy_client.cc:647] T b7c80c3b3a5d4505b7dbbac6bab1de59 P 09258ac095664fa99caf0707262cde13: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:34.925392 7418 tablet_copy_client.cc:523] T b7c80c3b3a5d4505b7dbbac6bab1de59 P 09258ac095664fa99caf0707262cde13: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:34.926815 7418 tablet_bootstrap.cc:492] T b7c80c3b3a5d4505b7dbbac6bab1de59 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:34.930768 29038 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:34.931885 7418 tablet_bootstrap.cc:492] T b7c80c3b3a5d4505b7dbbac6bab1de59 P 09258ac095664fa99caf0707262cde13: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:34.932111 7418 tablet_bootstrap.cc:492] T b7c80c3b3a5d4505b7dbbac6bab1de59 P 09258ac095664fa99caf0707262cde13: Bootstrap complete.
I20240502 20:46:34.932276 7418 ts_tablet_manager.cc:1367] T b7c80c3b3a5d4505b7dbbac6bab1de59 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20240502 20:46:34.932740 7418 raft_consensus.cc:348] T b7c80c3b3a5d4505b7dbbac6bab1de59 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:34.932847 7418 raft_consensus.cc:729] T b7c80c3b3a5d4505b7dbbac6bab1de59 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:34.938192 7170 tablet.cc:1722] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:34.947201 29315 raft_consensus.cc:1203] T 3eb48ef3d5404e5e9c302b904b5df2cd P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 20:46:34.948885 29038 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=e568822f039f4f97a3afbf915c8eaeca] alter complete (version 3)
I20240502 20:46:34.956506 7570 tablet.cc:1722] T 4432691bad6b42619ef72c41e72d27a9 P db17f26a73f3447b92aa9a947e3a7057: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:34.956921 7569 tablet.cc:1722] T 4432691bad6b42619ef72c41e72d27a9 P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:34.933954 7418 consensus_queue.cc:260] T b7c80c3b3a5d4505b7dbbac6bab1de59 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:34.971035 7418 ts_tablet_manager.cc:1398] T b7c80c3b3a5d4505b7dbbac6bab1de59 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.039s user 0.002s sys 0.001s
I20240502 20:46:34.973613 29465 tablet_copy_service.cc:342] P dbcdc6ac635b44b185c3eba63f66e9d6: Request end of tablet copy session 09258ac095664fa99caf0707262cde13-b7c80c3b3a5d4505b7dbbac6bab1de59 received from {username='slave'} at 127.22.45.66:57919
I20240502 20:46:34.973690 29465 tablet_copy_service.cc:433] P dbcdc6ac635b44b185c3eba63f66e9d6: ending tablet copy session 09258ac095664fa99caf0707262cde13-b7c80c3b3a5d4505b7dbbac6bab1de59 on tablet b7c80c3b3a5d4505b7dbbac6bab1de59 with peer 09258ac095664fa99caf0707262cde13
I20240502 20:46:34.988483 7683 ts_tablet_manager.cc:902] T 21ee3d7c6f764d0483b335da40e61efc P e7ab6b9053664ee5aa71c7978e26fe6a: Initiating tablet copy from peer 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:34.989665 7683 tablet_copy_client.cc:308] T 21ee3d7c6f764d0483b335da40e61efc P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Beginning tablet copy session from remote peer at address 127.22.45.69:43887
I20240502 20:46:34.991614 31515 tablet_copy_service.cc:140] P 2a9c1af8cf814030bde56b67ca5c2fbe: Received BeginTabletCopySession request for tablet 21ee3d7c6f764d0483b335da40e61efc from peer e7ab6b9053664ee5aa71c7978e26fe6a ({username='slave'} at 127.22.45.71:39583)
I20240502 20:46:34.991724 31515 tablet_copy_service.cc:161] P 2a9c1af8cf814030bde56b67ca5c2fbe: Beginning new tablet copy session on tablet 21ee3d7c6f764d0483b335da40e61efc from peer e7ab6b9053664ee5aa71c7978e26fe6a at {username='slave'} at 127.22.45.71:39583: session id = e7ab6b9053664ee5aa71c7978e26fe6a-21ee3d7c6f764d0483b335da40e61efc
I20240502 20:46:34.992731 31515 tablet_copy_source_session.cc:205] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 20:46:34.993582 7683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21ee3d7c6f764d0483b335da40e61efc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:34.999642 7683 tablet_copy_client.cc:783] T 21ee3d7c6f764d0483b335da40e61efc P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Starting download of 0 data blocks...
I20240502 20:46:34.999867 7683 tablet_copy_client.cc:647] T 21ee3d7c6f764d0483b335da40e61efc P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Starting download of 1 WAL segments...
I20240502 20:46:35.006624 31495 raft_consensus.cc:1203] T b28fe6bccf4f48a6a5456e6ead5cc920 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.5->[1.6-1.6] Dedup: 1.6->[]
I20240502 20:46:35.009622 7683 tablet_copy_client.cc:523] T 21ee3d7c6f764d0483b335da40e61efc P e7ab6b9053664ee5aa71c7978e26fe6a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 20:46:35.010921 7683 tablet_bootstrap.cc:492] T 21ee3d7c6f764d0483b335da40e61efc P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:35.018548 31757 raft_consensus.cc:1203] T 370bf1bcfb3343c488954ece2488b4c2 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 20:46:35.053885 31887 raft_consensus.cc:1203] T db9c5ce233da4b1a9ce70ae04aef7658 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.5->[1.6-1.6] Dedup: 1.6->[]
I20240502 20:46:35.062579 7683 tablet_bootstrap.cc:492] T 21ee3d7c6f764d0483b335da40e61efc P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 20:46:35.069478 29035 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:35.069609 29035 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:35.071817 29035 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 4432691bad6b42619ef72c41e72d27a9
I20240502 20:46:35.073323 29424 tablet_service.cc:1514] Processing DeleteTablet for tablet 4432691bad6b42619ef72c41e72d27a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:35 UTC) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:35.074268 29557 tablet_service.cc:1514] Processing DeleteTablet for tablet 4432691bad6b42619ef72c41e72d27a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:35 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:35.074586 7731 tablet_replica.cc:330] T 4432691bad6b42619ef72c41e72d27a9 P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:35.074573 7076 consensus_queue.cc:1035] T 3443c927080841c299ca239ac9eee2be P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:35.074728 7731 raft_consensus.cc:2229] T 4432691bad6b42619ef72c41e72d27a9 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:35.074837 7731 raft_consensus.cc:2258] T 4432691bad6b42619ef72c41e72d27a9 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:35.075318 7683 tablet_bootstrap.cc:492] T 21ee3d7c6f764d0483b335da40e61efc P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap complete.
I20240502 20:46:35.075675 7731 ts_tablet_manager.cc:1875] T 4432691bad6b42619ef72c41e72d27a9 P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:35.075991 7683 ts_tablet_manager.cc:1367] T 21ee3d7c6f764d0483b335da40e61efc P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.065s user 0.023s sys 0.005s
I20240502 20:46:35.076416 7683 raft_consensus.cc:348] T 21ee3d7c6f764d0483b335da40e61efc P e7ab6b9053664ee5aa71c7978e26fe6a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:35.076586 7683 raft_consensus.cc:729] T 21ee3d7c6f764d0483b335da40e61efc P e7ab6b9053664ee5aa71c7978e26fe6a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:35.076764 7683 consensus_queue.cc:260] T 21ee3d7c6f764d0483b335da40e61efc P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 2.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:35.077157 7683 ts_tablet_manager.cc:1398] T 21ee3d7c6f764d0483b335da40e61efc P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:35.078027 7731 ts_tablet_manager.cc:1888] T 4432691bad6b42619ef72c41e72d27a9 P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:35.078107 7731 log.cc:1196] T 4432691bad6b42619ef72c41e72d27a9 P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/4432691bad6b42619ef72c41e72d27a9
I20240502 20:46:35.078146 31515 tablet_copy_service.cc:342] P 2a9c1af8cf814030bde56b67ca5c2fbe: Request end of tablet copy session e7ab6b9053664ee5aa71c7978e26fe6a-21ee3d7c6f764d0483b335da40e61efc received from {username='slave'} at 127.22.45.71:39583
I20240502 20:46:35.078225 31515 tablet_copy_service.cc:433] P 2a9c1af8cf814030bde56b67ca5c2fbe: ending tablet copy session e7ab6b9053664ee5aa71c7978e26fe6a-21ee3d7c6f764d0483b335da40e61efc on tablet 21ee3d7c6f764d0483b335da40e61efc with peer e7ab6b9053664ee5aa71c7978e26fe6a
I20240502 20:46:35.078404 7731 ts_tablet_manager.cc:1909] T 4432691bad6b42619ef72c41e72d27a9 P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:35.079727 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet 4432691bad6b42619ef72c41e72d27a9 (table rebalancer_dynamic_table_new_name [id=e568822f039f4f97a3afbf915c8eaeca]) successfully deleted
I20240502 20:46:35.081354 6850 consensus_queue.cc:1035] T 3443c927080841c299ca239ac9eee2be P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:35.083370 29182 raft_consensus.cc:1203] T 21eab607fca143b8a165d777b50238e1 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 20:46:35.083699 31736 tablet_service.cc:1514] Processing DeleteTablet for tablet 4432691bad6b42619ef72c41e72d27a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:35 UTC) from {username='slave'} at 127.0.0.1:33864
I20240502 20:46:35.088841 7737 tablet_replica.cc:330] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a: stopping tablet replica
I20240502 20:46:35.089032 7737 raft_consensus.cc:2229] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:35.089267 7737 raft_consensus.cc:2258] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:35.090692 29446 raft_consensus.cc:1203] T 75ace61b6538462b9bd5c0820eeacca9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 20:46:35.096396 7737 ts_tablet_manager.cc:1875] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:35.099362 7737 ts_tablet_manager.cc:1888] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:35.099578 7737 log.cc:1196] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/wals/4432691bad6b42619ef72c41e72d27a9
I20240502 20:46:35.099893 7737 ts_tablet_manager.cc:1909] T 4432691bad6b42619ef72c41e72d27a9 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting consensus metadata
I20240502 20:46:35.100737 29026 catalog_manager.cc:4830] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): tablet 4432691bad6b42619ef72c41e72d27a9 (table rebalancer_dynamic_table_new_name [id=e568822f039f4f97a3afbf915c8eaeca]) successfully deleted
I20240502 20:46:35.101059 7730 tablet_replica.cc:330] T 4432691bad6b42619ef72c41e72d27a9 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:35.101292 7730 raft_consensus.cc:2229] T 4432691bad6b42619ef72c41e72d27a9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:35.101467 7730 raft_consensus.cc:2258] T 4432691bad6b42619ef72c41e72d27a9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:35.102082 7730 ts_tablet_manager.cc:1875] T 4432691bad6b42619ef72c41e72d27a9 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:35.104166 7730 ts_tablet_manager.cc:1888] T 4432691bad6b42619ef72c41e72d27a9 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:35.104243 7730 log.cc:1196] T 4432691bad6b42619ef72c41e72d27a9 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/4432691bad6b42619ef72c41e72d27a9
I20240502 20:46:35.104583 7730 ts_tablet_manager.cc:1909] T 4432691bad6b42619ef72c41e72d27a9 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting consensus metadata
I20240502 20:46:35.105861 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet 4432691bad6b42619ef72c41e72d27a9 (table rebalancer_dynamic_table_new_name [id=e568822f039f4f97a3afbf915c8eaeca]) successfully deleted
W20240502 20:46:35.111652 29118 connection.cc:535] client connection to 127.22.45.70:46751 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 20:46:35.111793 29118 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 20:46:35.111838 29511 connection.cc:535] client connection to 127.22.45.70:46751 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 20:46:35.111946 29511 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 20:46:35.112671 31429 connection.cc:535] client connection to 127.22.45.70:46751 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 20:46:35.112682 29380 connection.cc:535] client connection to 127.22.45.70:46751 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 20:46:35.112782 29380 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 20:46:35.112787 31429 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 20:46:35.113518 31691 connection.cc:535] client connection to 127.22.45.70:46751 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 20:46:35.113497 29249 connection.cc:535] client connection to 127.22.45.70:46751 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 20:46:35.113620 29249 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 20:46:35.113633 31691 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 20:46:35.115118 7757 negotiation.cc:337] Failed RPC negotiation. Trace:
0502 20:46:35.113979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.45.70:46751 (local address 127.22.45.69:51355)
0502 20:46:35.114691 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0502 20:46:35.114750 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":56,"threads_started":1}
W20240502 20:46:35.115118 7750 negotiation.cc:337] Failed RPC negotiation. Trace:
0502 20:46:35.112399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.45.70:46751 (local address 127.22.45.65:39165)
0502 20:46:35.113018 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0502 20:46:35.113032 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0502 20:46:35.113250 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0502 20:46:35.114587 (+ 1337us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":49,"threads_started":1}
W20240502 20:46:35.115249 29118 consensus_peers.cc:485] T 3d89ea92aec54168997241dd78c6abb1 P 7796ea3fba294d17a34386873969431d -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:35.115371 31429 consensus_peers.cc:485] T 4afde5c1bf754a819f84bcc41212617d P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection reset by peer (error 104). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:35.115428 31429 consensus_peers.cc:485] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection reset by peer (error 104). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:35.115832 29380 consensus_peers.cc:485] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:35.115850 7754 negotiation.cc:337] Failed RPC negotiation. Trace:
0502 20:46:35.113025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.45.70:46751 (local address 127.22.45.68:38993)
0502 20:46:35.113520 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0502 20:46:35.113531 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0502 20:46:35.113606 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0502 20:46:35.115371 (+ 1765us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":55,"threads_started":1}
W20240502 20:46:35.115907 29380 consensus_peers.cc:485] T 653cbec396434b4dac0a1928b21d0ebd P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:35.115954 29511 consensus_peers.cc:485] T 9da27cc9c2504b7899da253a88dd1f56 P db17f26a73f3447b92aa9a947e3a7057 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:35.116004 29511 consensus_peers.cc:485] T db5e418e67a44480b8d4c302cde3b96c P db17f26a73f3447b92aa9a947e3a7057 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:35.117120 31691 consensus_peers.cc:485] T d6b1c1798aaf4f1fb3d6ef52503e08be P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:35.117194 31691 consensus_peers.cc:485] T 46c0a5b0870c4a099c7c17ebc0a5d34e P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:35.117208 31429 consensus_peers.cc:485] T d0538f41ea0e46edae87b8e4dfa995f0 P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:35.117231 31691 consensus_peers.cc:462] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
W20240502 20:46:35.117260 31429 consensus_peers.cc:485] T 211fa6a23bda4c11bb3106080c3ace40 P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:35.117297 31429 consensus_peers.cc:485] T 2e4a0e5ef89d447b96b998f085e3495c P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:35.117377 29249 consensus_peers.cc:485] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:35.117444 29249 consensus_peers.cc:485] T f3c220fbd7df4cefb467ad88e99941b8 P 09258ac095664fa99caf0707262cde13 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:35.127986 31822 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
W20240502 20:46:35.128865 31822 consensus_peers.cc:485] T 8feefea4624e49309cef9ecb952bbe96 P e6b60803d0764d228c863e2da65e3691 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:35.132302 31429 consensus_peers.cc:485] T b718cecdc692408bbeea91bc9fe21fcb P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:35.145555 29249 consensus_peers.cc:485] T 9992a7ea196645dc82d2430fcb405254 P 09258ac095664fa99caf0707262cde13 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:35.147073 29380 consensus_peers.cc:485] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:35.154989 31691 consensus_peers.cc:485] T 6436a7d43dab46bca948438f832245ed P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:35.156013 31691 consensus_peers.cc:485] T 1b2fc2ad5cd24a128562714712e89d89 P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:35.173355 29118 consensus_peers.cc:485] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:35.173833 29038 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:35.179242 29557 tablet_service.cc:1467] Processing CreateTablet for tablet e0bf2fff32214e349ae0cf765f7343e1 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=4b3014b5ed314186914b7365db776294]), partition=
I20240502 20:46:35.179481 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0bf2fff32214e349ae0cf765f7343e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:35.179708 31868 tablet_service.cc:1467] Processing CreateTablet for tablet e0bf2fff32214e349ae0cf765f7343e1 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=4b3014b5ed314186914b7365db776294]), partition=
I20240502 20:46:35.179935 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0bf2fff32214e349ae0cf765f7343e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:35.181880 31736 tablet_service.cc:1467] Processing CreateTablet for tablet e0bf2fff32214e349ae0cf765f7343e1 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=4b3014b5ed314186914b7365db776294]), partition=
I20240502 20:46:35.181972 7793 tablet_bootstrap.cc:492] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:35.182101 31736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0bf2fff32214e349ae0cf765f7343e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:35.183497 7793 tablet_bootstrap.cc:654] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:35.183822 29315 raft_consensus.cc:1203] T 2bec922ff95845d1a13c27526f689e15 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 20:46:35.184629 7795 tablet_bootstrap.cc:492] T e0bf2fff32214e349ae0cf765f7343e1 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:35.185245 7794 tablet_bootstrap.cc:492] T e0bf2fff32214e349ae0cf765f7343e1 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
W20240502 20:46:35.186395 29380 consensus_peers.cc:485] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:35.187150 7795 tablet_bootstrap.cc:654] T e0bf2fff32214e349ae0cf765f7343e1 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:35.187497 7794 tablet_bootstrap.cc:654] T e0bf2fff32214e349ae0cf765f7343e1 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:35.188091 7022 consensus_queue.cc:1035] T 5468fdb5d2a240acbda0546d54b46b74 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 20:46:35.188813 29118 consensus_peers.cc:485] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:35.189074 7795 tablet_bootstrap.cc:492] T e0bf2fff32214e349ae0cf765f7343e1 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:35.189146 7795 ts_tablet_manager.cc:1367] T e0bf2fff32214e349ae0cf765f7343e1 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20240502 20:46:35.189551 7795 raft_consensus.cc:348] T e0bf2fff32214e349ae0cf765f7343e1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:35.189640 7795 raft_consensus.cc:374] T e0bf2fff32214e349ae0cf765f7343e1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:35.189669 7795 raft_consensus.cc:729] T e0bf2fff32214e349ae0cf765f7343e1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:35.190922 7795 consensus_queue.cc:260] T e0bf2fff32214e349ae0cf765f7343e1 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:35.191311 7795 ts_tablet_manager.cc:1398] T e0bf2fff32214e349ae0cf765f7343e1 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:35.191910 7794 tablet_bootstrap.cc:492] T e0bf2fff32214e349ae0cf765f7343e1 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:35.192049 7794 ts_tablet_manager.cc:1367] T e0bf2fff32214e349ae0cf765f7343e1 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 20:46:35.192468 7794 raft_consensus.cc:348] T e0bf2fff32214e349ae0cf765f7343e1 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:35.192608 7794 raft_consensus.cc:374] T e0bf2fff32214e349ae0cf765f7343e1 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:35.192687 7794 raft_consensus.cc:729] T e0bf2fff32214e349ae0cf765f7343e1 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:35.193064 7794 consensus_queue.cc:260] T e0bf2fff32214e349ae0cf765f7343e1 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:35.193408 7794 ts_tablet_manager.cc:1398] T e0bf2fff32214e349ae0cf765f7343e1 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:35.195709 7022 consensus_queue.cc:1035] T 5468fdb5d2a240acbda0546d54b46b74 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 20:46:35.205276 29511 consensus_peers.cc:485] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:35.206954 7793 tablet_bootstrap.cc:492] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:35.207108 7793 ts_tablet_manager.cc:1367] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20240502 20:46:35.208110 7793 raft_consensus.cc:348] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:35.208271 7793 raft_consensus.cc:374] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:35.208350 7793 raft_consensus.cc:729] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:35.208557 7793 consensus_queue.cc:260] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:35.214645 7793 ts_tablet_manager.cc:1398] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
W20240502 20:46:35.215970 29511 consensus_peers.cc:485] T b9c1ec018c544b3bb90fe57fe9995920 P db17f26a73f3447b92aa9a947e3a7057 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:35.223356 29380 consensus_peers.cc:485] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:35.228372 31822 consensus_peers.cc:485] T 63487637b76a4de89ac3f53c707f074b P e6b60803d0764d228c863e2da65e3691 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:35.236007 7076 consensus_queue.cc:1035] T 8feefea4624e49309cef9ecb952bbe96 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:35.241529 29038 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_58"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:35.242813 29295 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:35.247278 29315 raft_consensus.cc:1203] T 7db3d66689ef4fb2869e94448515ad12 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 20:46:35.249006 31474 tablet_service.cc:1467] Processing CreateTablet for tablet bcab2a7cb1974024899254facc6af322 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=cd0fc1b937cc460397b51cfd94ffdda9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:35.249269 31474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcab2a7cb1974024899254facc6af322. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:35.250015 31736 tablet_service.cc:1467] Processing CreateTablet for tablet 1202e6d31c4a4f0895b40002ac1d3dad (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=cd0fc1b937cc460397b51cfd94ffdda9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:35.250276 31736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1202e6d31c4a4f0895b40002ac1d3dad. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:35.251788 7819 tablet_bootstrap.cc:492] T bcab2a7cb1974024899254facc6af322 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:35.251760 31868 tablet_service.cc:1467] Processing CreateTablet for tablet 1202e6d31c4a4f0895b40002ac1d3dad (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=cd0fc1b937cc460397b51cfd94ffdda9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:35.251994 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1202e6d31c4a4f0895b40002ac1d3dad. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:35.252701 7819 tablet_bootstrap.cc:654] T bcab2a7cb1974024899254facc6af322 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:35.252991 31475 tablet_service.cc:1467] Processing CreateTablet for tablet da2da056404f4be5b9ed325bac7ddcb2 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=cd0fc1b937cc460397b51cfd94ffdda9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:35.253194 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da2da056404f4be5b9ed325bac7ddcb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:35.254654 7795 tablet_bootstrap.cc:492] T 1202e6d31c4a4f0895b40002ac1d3dad P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:35.255522 7819 tablet_bootstrap.cc:492] T bcab2a7cb1974024899254facc6af322 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:35.255556 7795 tablet_bootstrap.cc:654] T 1202e6d31c4a4f0895b40002ac1d3dad P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:35.255599 7819 ts_tablet_manager.cc:1367] T bcab2a7cb1974024899254facc6af322 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:35.256275 31735 tablet_service.cc:1467] Processing CreateTablet for tablet bcab2a7cb1974024899254facc6af322 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=cd0fc1b937cc460397b51cfd94ffdda9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:35.256505 31735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcab2a7cb1974024899254facc6af322. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:35.256851 7795 tablet_bootstrap.cc:492] T 1202e6d31c4a4f0895b40002ac1d3dad P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:35.256914 7795 ts_tablet_manager.cc:1367] T 1202e6d31c4a4f0895b40002ac1d3dad P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 20:46:35.257336 7795 raft_consensus.cc:348] T 1202e6d31c4a4f0895b40002ac1d3dad P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:35.257442 7795 raft_consensus.cc:374] T 1202e6d31c4a4f0895b40002ac1d3dad P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:35.257477 7795 raft_consensus.cc:729] T 1202e6d31c4a4f0895b40002ac1d3dad P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:35.257584 7795 consensus_queue.cc:260] T 1202e6d31c4a4f0895b40002ac1d3dad P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:35.257881 7795 ts_tablet_manager.cc:1398] T 1202e6d31c4a4f0895b40002ac1d3dad P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:35.259232 29294 tablet_service.cc:1467] Processing CreateTablet for tablet da2da056404f4be5b9ed325bac7ddcb2 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=cd0fc1b937cc460397b51cfd94ffdda9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:35.259500 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da2da056404f4be5b9ed325bac7ddcb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:35.259233 29295 tablet_service.cc:1467] Processing CreateTablet for tablet bcab2a7cb1974024899254facc6af322 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=cd0fc1b937cc460397b51cfd94ffdda9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:35.260257 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcab2a7cb1974024899254facc6af322. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:35.263177 7819 raft_consensus.cc:348] T bcab2a7cb1974024899254facc6af322 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:35.263306 7819 raft_consensus.cc:374] T bcab2a7cb1974024899254facc6af322 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:35.263332 7819 raft_consensus.cc:729] T bcab2a7cb1974024899254facc6af322 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:35.263434 7819 consensus_queue.cc:260] T bcab2a7cb1974024899254facc6af322 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:35.263576 7795 tablet_bootstrap.cc:492] T bcab2a7cb1974024899254facc6af322 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:35.264467 7795 tablet_bootstrap.cc:654] T bcab2a7cb1974024899254facc6af322 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:35.265190 7822 tablet_bootstrap.cc:492] T da2da056404f4be5b9ed325bac7ddcb2 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:35.266626 29557 tablet_service.cc:1467] Processing CreateTablet for tablet 1202e6d31c4a4f0895b40002ac1d3dad (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=cd0fc1b937cc460397b51cfd94ffdda9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:35.266878 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1202e6d31c4a4f0895b40002ac1d3dad. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:35.268406 29446 raft_consensus.cc:1203] T 43fc3dcc2dd54a11becb8f16ec500df1 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 20:46:35.269351 7793 tablet_bootstrap.cc:492] T 1202e6d31c4a4f0895b40002ac1d3dad P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:35.270434 7793 tablet_bootstrap.cc:654] T 1202e6d31c4a4f0895b40002ac1d3dad P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
W20240502 20:46:35.270793 31691 consensus_peers.cc:462] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:35.271045 7794 tablet_bootstrap.cc:492] T 1202e6d31c4a4f0895b40002ac1d3dad P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:35.271325 31735 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:35.271744 7822 tablet_bootstrap.cc:654] T da2da056404f4be5b9ed325bac7ddcb2 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:35.271776 7794 tablet_bootstrap.cc:654] T 1202e6d31c4a4f0895b40002ac1d3dad P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:35.272753 7822 tablet_bootstrap.cc:492] T da2da056404f4be5b9ed325bac7ddcb2 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:35.272835 7822 ts_tablet_manager.cc:1367] T da2da056404f4be5b9ed325bac7ddcb2 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 20:46:35.273270 7822 raft_consensus.cc:348] T da2da056404f4be5b9ed325bac7ddcb2 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:35.273365 7822 raft_consensus.cc:374] T da2da056404f4be5b9ed325bac7ddcb2 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:35.273396 7822 raft_consensus.cc:729] T da2da056404f4be5b9ed325bac7ddcb2 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:35.273515 7822 consensus_queue.cc:260] T da2da056404f4be5b9ed325bac7ddcb2 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:35.273785 7822 ts_tablet_manager.cc:1398] T da2da056404f4be5b9ed325bac7ddcb2 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:35.273928 7822 tablet_bootstrap.cc:492] T bcab2a7cb1974024899254facc6af322 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
W20240502 20:46:35.274121 29511 consensus_peers.cc:485] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:35.274168 29557 tablet_service.cc:1467] Processing CreateTablet for tablet da2da056404f4be5b9ed325bac7ddcb2 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=cd0fc1b937cc460397b51cfd94ffdda9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:35.274377 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da2da056404f4be5b9ed325bac7ddcb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:35.274744 7822 tablet_bootstrap.cc:654] T bcab2a7cb1974024899254facc6af322 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:35.275988 7819 ts_tablet_manager.cc:1398] T bcab2a7cb1974024899254facc6af322 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.020s user 0.001s sys 0.000s
I20240502 20:46:35.276685 29556 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
W20240502 20:46:35.277993 31429 consensus_peers.cc:485] T b699c00ff92b41e9ae24e5982ced3ef9 P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:35.278061 31429 consensus_peers.cc:485] T 2a00833b5b404e63b0f720b5c8fb528d P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:35.279546 7819 tablet_bootstrap.cc:492] T da2da056404f4be5b9ed325bac7ddcb2 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:35.281833 29175 raft_consensus.cc:1203] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 20:46:35.282490 29163 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 20:46:35.283767 7819 tablet_bootstrap.cc:654] T da2da056404f4be5b9ed325bac7ddcb2 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:35.284080 29574 raft_consensus.cc:1203] T d5ca50fc288f4957a2d6f04c23bd1491 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 20:46:35.284795 7795 tablet_bootstrap.cc:492] T bcab2a7cb1974024899254facc6af322 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:35.284878 7795 ts_tablet_manager.cc:1367] T bcab2a7cb1974024899254facc6af322 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.002s
I20240502 20:46:35.285305 7795 raft_consensus.cc:348] T bcab2a7cb1974024899254facc6af322 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:35.285398 7795 raft_consensus.cc:374] T bcab2a7cb1974024899254facc6af322 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:35.285449 7795 raft_consensus.cc:729] T bcab2a7cb1974024899254facc6af322 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:35.285552 7795 consensus_queue.cc:260] T bcab2a7cb1974024899254facc6af322 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:35.285815 7795 ts_tablet_manager.cc:1398] T bcab2a7cb1974024899254facc6af322 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:35.286042 29424 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 20:46:35.286238 7794 tablet_bootstrap.cc:492] T 1202e6d31c4a4f0895b40002ac1d3dad P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:35.286310 7794 ts_tablet_manager.cc:1367] T 1202e6d31c4a4f0895b40002ac1d3dad P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20240502 20:46:35.286919 7799 raft_consensus.cc:482] T bcab2a7cb1974024899254facc6af322 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:35.286996 7799 raft_consensus.cc:504] T bcab2a7cb1974024899254facc6af322 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:35.287338 7799 leader_election.cc:288] T bcab2a7cb1974024899254facc6af322 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:35.287747 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcab2a7cb1974024899254facc6af322" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:35.288126 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcab2a7cb1974024899254facc6af322" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
W20240502 20:46:35.288187 31430 leader_election.cc:341] T bcab2a7cb1974024899254facc6af322 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): Illegal state: must be running to vote when last-logged opid is not known
I20240502 20:46:35.288230 31755 raft_consensus.cc:2454] T bcab2a7cb1974024899254facc6af322 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:35.288568 7794 raft_consensus.cc:348] T 1202e6d31c4a4f0895b40002ac1d3dad P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:35.288659 7794 raft_consensus.cc:374] T 1202e6d31c4a4f0895b40002ac1d3dad P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:35.288686 7794 raft_consensus.cc:729] T 1202e6d31c4a4f0895b40002ac1d3dad P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:35.288784 7794 consensus_queue.cc:260] T 1202e6d31c4a4f0895b40002ac1d3dad P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:35.288785 31428 leader_election.cc:302] T bcab2a7cb1974024899254facc6af322 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e7ab6b9053664ee5aa71c7978e26fe6a; no voters: 09258ac095664fa99caf0707262cde13
I20240502 20:46:35.289031 7794 ts_tablet_manager.cc:1398] T 1202e6d31c4a4f0895b40002ac1d3dad P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:35.289530 7022 raft_consensus.cc:2781] T bcab2a7cb1974024899254facc6af322 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:35.289578 7022 raft_consensus.cc:482] T bcab2a7cb1974024899254facc6af322 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:35.289611 7022 raft_consensus.cc:3037] T bcab2a7cb1974024899254facc6af322 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:35.290536 7022 raft_consensus.cc:504] T bcab2a7cb1974024899254facc6af322 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:35.290851 7022 leader_election.cc:288] T bcab2a7cb1974024899254facc6af322 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Requested vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:35.290920 7819 tablet_bootstrap.cc:492] T da2da056404f4be5b9ed325bac7ddcb2 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:35.290999 7819 ts_tablet_manager.cc:1367] T da2da056404f4be5b9ed325bac7ddcb2 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240502 20:46:35.291363 29313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcab2a7cb1974024899254facc6af322" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:35.291419 7819 raft_consensus.cc:348] T da2da056404f4be5b9ed325bac7ddcb2 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:35.291504 7819 raft_consensus.cc:374] T da2da056404f4be5b9ed325bac7ddcb2 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:35.291533 7819 raft_consensus.cc:729] T da2da056404f4be5b9ed325bac7ddcb2 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:35.291628 7819 consensus_queue.cc:260] T da2da056404f4be5b9ed325bac7ddcb2 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:35.291716 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcab2a7cb1974024899254facc6af322" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
W20240502 20:46:35.291730 31430 leader_election.cc:341] T bcab2a7cb1974024899254facc6af322 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): Illegal state: must be running to vote when last-logged opid is not known
I20240502 20:46:35.291801 31755 raft_consensus.cc:3037] T bcab2a7cb1974024899254facc6af322 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:35.292196 7819 ts_tablet_manager.cc:1398] T da2da056404f4be5b9ed325bac7ddcb2 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:35.292737 31755 raft_consensus.cc:2454] T bcab2a7cb1974024899254facc6af322 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:35.295331 31495 raft_consensus.cc:1203] T c8dc196b2d4e48b195524bc79c07b9eb P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 20:46:35.295877 31428 leader_election.cc:302] T bcab2a7cb1974024899254facc6af322 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e7ab6b9053664ee5aa71c7978e26fe6a; no voters: 09258ac095664fa99caf0707262cde13
I20240502 20:46:35.296818 7799 raft_consensus.cc:2781] T bcab2a7cb1974024899254facc6af322 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:35.296896 7799 raft_consensus.cc:686] T bcab2a7cb1974024899254facc6af322 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Becoming Leader. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Running, Role: LEADER
W20240502 20:46:35.296955 31822 consensus_peers.cc:485] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:35.297005 7799 consensus_queue.cc:237] T bcab2a7cb1974024899254facc6af322 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:35.297482 31868 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:35.300510 29039 catalog_manager.cc:5484] T bcab2a7cb1974024899254facc6af322 P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69). New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
W20240502 20:46:35.306631 29249 consensus_peers.cc:485] T f0188e48270242949913034b04dbd3b0 P 09258ac095664fa99caf0707262cde13 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:35.307471 7022 raft_consensus.cc:482] T da2da056404f4be5b9ed325bac7ddcb2 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:35.307545 7022 raft_consensus.cc:504] T da2da056404f4be5b9ed325bac7ddcb2 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:35.307919 7022 leader_election.cc:288] T da2da056404f4be5b9ed325bac7ddcb2 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:35.310859 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da2da056404f4be5b9ed325bac7ddcb2" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:35.311115 31475 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 20:46:35.312086 29313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da2da056404f4be5b9ed325bac7ddcb2" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
W20240502 20:46:35.312119 31429 leader_election.cc:341] T da2da056404f4be5b9ed325bac7ddcb2 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): Illegal state: must be running to vote when last-logged opid is not known
I20240502 20:46:35.312223 29313 raft_consensus.cc:2454] T da2da056404f4be5b9ed325bac7ddcb2 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:35.312502 31430 leader_election.cc:302] T da2da056404f4be5b9ed325bac7ddcb2 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 2a9c1af8cf814030bde56b67ca5c2fbe; no voters: db17f26a73f3447b92aa9a947e3a7057
I20240502 20:46:35.312786 7022 raft_consensus.cc:2781] T da2da056404f4be5b9ed325bac7ddcb2 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:35.312830 7022 raft_consensus.cc:482] T da2da056404f4be5b9ed325bac7ddcb2 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:35.312877 7022 raft_consensus.cc:3037] T da2da056404f4be5b9ed325bac7ddcb2 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:35.313757 7022 raft_consensus.cc:504] T da2da056404f4be5b9ed325bac7ddcb2 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:35.314361 7022 leader_election.cc:288] T da2da056404f4be5b9ed325bac7ddcb2 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:35.315052 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da2da056404f4be5b9ed325bac7ddcb2" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:35.315553 29313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da2da056404f4be5b9ed325bac7ddcb2" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:35.315658 29313 raft_consensus.cc:3037] T da2da056404f4be5b9ed325bac7ddcb2 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:35.316524 29313 raft_consensus.cc:2454] T da2da056404f4be5b9ed325bac7ddcb2 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:35.316804 31430 leader_election.cc:302] T da2da056404f4be5b9ed325bac7ddcb2 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 2a9c1af8cf814030bde56b67ca5c2fbe; no voters:
I20240502 20:46:35.316963 7022 raft_consensus.cc:2781] T da2da056404f4be5b9ed325bac7ddcb2 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:35.317381 7022 raft_consensus.cc:686] T da2da056404f4be5b9ed325bac7ddcb2 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Becoming Leader. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Running, Role: LEADER
I20240502 20:46:35.317695 7022 consensus_queue.cc:237] T da2da056404f4be5b9ed325bac7ddcb2 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
W20240502 20:46:35.318189 31822 consensus_peers.cc:485] T 7a8b04cb606f423f8dd1d489d47408d2 P e6b60803d0764d228c863e2da65e3691 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:35.318687 31429 leader_election.cc:341] T da2da056404f4be5b9ed325bac7ddcb2 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): Illegal state: must be running to vote when last-logged opid is not known
I20240502 20:46:35.320040 29039 catalog_manager.cc:5484] T da2da056404f4be5b9ed325bac7ddcb2 P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69). New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:35.322609 7256 raft_consensus.cc:482] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:35.322695 7256 raft_consensus.cc:504] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:35.323024 7256 leader_election.cc:288] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:35.323347 31887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0bf2fff32214e349ae0cf765f7343e1" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:35.323450 31887 raft_consensus.cc:2454] T e0bf2fff32214e349ae0cf765f7343e1 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:35.323794 7793 tablet_bootstrap.cc:492] T 1202e6d31c4a4f0895b40002ac1d3dad P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:35.323869 7793 ts_tablet_manager.cc:1367] T 1202e6d31c4a4f0895b40002ac1d3dad P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.055s user 0.002s sys 0.000s
I20240502 20:46:35.324092 29511 leader_election.cc:302] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:35.324311 7793 raft_consensus.cc:348] T 1202e6d31c4a4f0895b40002ac1d3dad P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:35.324415 7793 raft_consensus.cc:374] T 1202e6d31c4a4f0895b40002ac1d3dad P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:35.324441 7793 raft_consensus.cc:729] T 1202e6d31c4a4f0895b40002ac1d3dad P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:35.324543 7793 consensus_queue.cc:260] T 1202e6d31c4a4f0895b40002ac1d3dad P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:35.324807 7793 ts_tablet_manager.cc:1398] T 1202e6d31c4a4f0895b40002ac1d3dad P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:35.324873 7256 raft_consensus.cc:2781] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:35.324937 7256 raft_consensus.cc:482] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:35.324970 7256 raft_consensus.cc:3037] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:35.324997 7793 tablet_bootstrap.cc:492] T da2da056404f4be5b9ed325bac7ddcb2 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:35.325953 7793 tablet_bootstrap.cc:654] T da2da056404f4be5b9ed325bac7ddcb2 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:35.325974 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0bf2fff32214e349ae0cf765f7343e1" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:35.326081 31757 raft_consensus.cc:2454] T e0bf2fff32214e349ae0cf765f7343e1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:35.325963 7256 raft_consensus.cc:504] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:35.326898 7256 leader_election.cc:288] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Requested vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:35.327358 31887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0bf2fff32214e349ae0cf765f7343e1" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:35.327451 31887 raft_consensus.cc:3037] T e0bf2fff32214e349ae0cf765f7343e1 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:35.328317 31887 raft_consensus.cc:2454] T e0bf2fff32214e349ae0cf765f7343e1 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:35.328568 29511 leader_election.cc:302] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:35.328805 7256 raft_consensus.cc:2781] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:35.328886 7256 raft_consensus.cc:686] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Becoming Leader. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Running, Role: LEADER
I20240502 20:46:35.328967 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0bf2fff32214e349ae0cf765f7343e1" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:35.328995 7256 consensus_queue.cc:237] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:35.329062 31757 raft_consensus.cc:3037] T e0bf2fff32214e349ae0cf765f7343e1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:35.329938 31757 raft_consensus.cc:2454] T e0bf2fff32214e349ae0cf765f7343e1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:35.334342 29035 catalog_manager.cc:5484] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: term changed from 0 to 1, leader changed from <none> to db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:35.339190 7822 tablet_bootstrap.cc:492] T bcab2a7cb1974024899254facc6af322 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:35.339288 7822 ts_tablet_manager.cc:1367] T bcab2a7cb1974024899254facc6af322 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.065s user 0.002s sys 0.000s
I20240502 20:46:35.339879 7822 raft_consensus.cc:348] T bcab2a7cb1974024899254facc6af322 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:35.340050 7822 raft_consensus.cc:374] T bcab2a7cb1974024899254facc6af322 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:35.340109 7822 raft_consensus.cc:729] T bcab2a7cb1974024899254facc6af322 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:35.340266 7822 consensus_queue.cc:260] T bcab2a7cb1974024899254facc6af322 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:35.340742 7822 ts_tablet_manager.cc:1398] T bcab2a7cb1974024899254facc6af322 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:35.341711 7793 tablet_bootstrap.cc:492] T da2da056404f4be5b9ed325bac7ddcb2 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:35.341786 7793 ts_tablet_manager.cc:1367] T da2da056404f4be5b9ed325bac7ddcb2 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.001s
I20240502 20:46:35.342187 7793 raft_consensus.cc:348] T da2da056404f4be5b9ed325bac7ddcb2 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:35.342270 7793 raft_consensus.cc:374] T da2da056404f4be5b9ed325bac7ddcb2 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:35.342299 7793 raft_consensus.cc:729] T da2da056404f4be5b9ed325bac7ddcb2 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:35.342396 7793 consensus_queue.cc:260] T da2da056404f4be5b9ed325bac7ddcb2 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:35.342671 7793 ts_tablet_manager.cc:1398] T da2da056404f4be5b9ed325bac7ddcb2 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:35.344373 29313 raft_consensus.cc:1203] T b7c80c3b3a5d4505b7dbbac6bab1de59 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 20:46:35.374083 31757 raft_consensus.cc:1203] T 21ee3d7c6f764d0483b335da40e61efc P e7ab6b9053664ee5aa71c7978e26fe6a [term 2 FOLLOWER]: Deduplicated request from leader. Original: 2.6->[2.7-2.7] Dedup: 2.7->[]
I20240502 20:46:35.397601 7181 raft_consensus.cc:482] T bcab2a7cb1974024899254facc6af322 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:35.397692 7181 raft_consensus.cc:504] T bcab2a7cb1974024899254facc6af322 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:35.398031 7181 leader_election.cc:288] T bcab2a7cb1974024899254facc6af322 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:35.398546 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcab2a7cb1974024899254facc6af322" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:35.399379 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcab2a7cb1974024899254facc6af322" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:35.399576 31755 raft_consensus.cc:2379] T bcab2a7cb1974024899254facc6af322 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 09258ac095664fa99caf0707262cde13 in current term 1: Already voted for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in this term.
I20240502 20:46:35.399854 29248 leader_election.cc:302] T bcab2a7cb1974024899254facc6af322 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 09258ac095664fa99caf0707262cde13; no voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e7ab6b9053664ee5aa71c7978e26fe6a
I20240502 20:46:35.400640 7181 raft_consensus.cc:3037] T bcab2a7cb1974024899254facc6af322 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:35.401661 7181 raft_consensus.cc:2726] T bcab2a7cb1974024899254facc6af322 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20240502 20:46:35.403817 29380 consensus_peers.cc:485] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:35.403849 31822 consensus_peers.cc:485] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:35.457216 29035 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:35.464829 31755 raft_consensus.cc:1261] T e0bf2fff32214e349ae0cf765f7343e1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:35.464843 31887 raft_consensus.cc:1261] T e0bf2fff32214e349ae0cf765f7343e1 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:35.465173 5346 consensus_queue.cc:1035] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:35.465413 7256 consensus_queue.cc:1035] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:35.468438 7878 tablet.cc:1722] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:35.468788 7878 tablet.cc:1620] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057: MemRowSet was empty: no flush needed.
I20240502 20:46:35.469045 29034 catalog_manager.cc:5576] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207) does not have the latest schema for tablet e0bf2fff32214e349ae0cf765f7343e1 (table rebalancer_dynamic_table [id=4b3014b5ed314186914b7365db776294]). Expected version 1 got 0
I20240502 20:46:35.470067 29033 catalog_manager.cc:5576] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787) does not have the latest schema for tablet e0bf2fff32214e349ae0cf765f7343e1 (table rebalancer_dynamic_table [id=4b3014b5ed314186914b7365db776294]). Expected version 1 got 0
I20240502 20:46:35.470470 7880 tablet.cc:1722] T e0bf2fff32214e349ae0cf765f7343e1 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:35.470758 7880 tablet.cc:1620] T e0bf2fff32214e349ae0cf765f7343e1 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:35.474279 7881 tablet.cc:1722] T e0bf2fff32214e349ae0cf765f7343e1 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:35.475108 7881 tablet.cc:1620] T e0bf2fff32214e349ae0cf765f7343e1 P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:35.474846 29033 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=4b3014b5ed314186914b7365db776294] alter complete (version 1)
I20240502 20:46:35.476071 29035 catalog_manager.cc:5576] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787) does not have the latest schema for tablet e0bf2fff32214e349ae0cf765f7343e1 (table rebalancer_dynamic_table [id=4b3014b5ed314186914b7365db776294]). Expected version 1 got 0
I20240502 20:46:35.529464 7200 raft_consensus.cc:482] T 1202e6d31c4a4f0895b40002ac1d3dad P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:35.529620 7200 raft_consensus.cc:504] T 1202e6d31c4a4f0895b40002ac1d3dad P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:35.530018 7200 leader_election.cc:288] T 1202e6d31c4a4f0895b40002ac1d3dad P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:35.530269 31887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1202e6d31c4a4f0895b40002ac1d3dad" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:35.530388 31887 raft_consensus.cc:2454] T 1202e6d31c4a4f0895b40002ac1d3dad P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:35.530366 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1202e6d31c4a4f0895b40002ac1d3dad" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:35.530534 29574 raft_consensus.cc:2454] T 1202e6d31c4a4f0895b40002ac1d3dad P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:35.530664 31691 leader_election.cc:302] T 1202e6d31c4a4f0895b40002ac1d3dad P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e6b60803d0764d228c863e2da65e3691, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:35.530843 7200 raft_consensus.cc:2781] T 1202e6d31c4a4f0895b40002ac1d3dad P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:35.530905 7200 raft_consensus.cc:482] T 1202e6d31c4a4f0895b40002ac1d3dad P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:35.530956 7200 raft_consensus.cc:3037] T 1202e6d31c4a4f0895b40002ac1d3dad P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:35.532047 7200 raft_consensus.cc:504] T 1202e6d31c4a4f0895b40002ac1d3dad P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:35.532413 7200 leader_election.cc:288] T 1202e6d31c4a4f0895b40002ac1d3dad P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Requested vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:35.532754 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1202e6d31c4a4f0895b40002ac1d3dad" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:35.532850 29574 raft_consensus.cc:3037] T 1202e6d31c4a4f0895b40002ac1d3dad P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:35.533627 29574 raft_consensus.cc:2454] T 1202e6d31c4a4f0895b40002ac1d3dad P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:35.533836 31887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1202e6d31c4a4f0895b40002ac1d3dad" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:35.533917 31691 leader_election.cc:302] T 1202e6d31c4a4f0895b40002ac1d3dad P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:35.533932 31887 raft_consensus.cc:3037] T 1202e6d31c4a4f0895b40002ac1d3dad P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:35.534096 7200 raft_consensus.cc:2781] T 1202e6d31c4a4f0895b40002ac1d3dad P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:35.534309 7200 raft_consensus.cc:686] T 1202e6d31c4a4f0895b40002ac1d3dad P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Becoming Leader. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Running, Role: LEADER
I20240502 20:46:35.534577 7200 consensus_queue.cc:237] T 1202e6d31c4a4f0895b40002ac1d3dad P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:35.534796 31887 raft_consensus.cc:2454] T 1202e6d31c4a4f0895b40002ac1d3dad P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:35.537142 29033 catalog_manager.cc:5484] T 1202e6d31c4a4f0895b40002ac1d3dad P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: term changed from 0 to 1, leader changed from <none> to e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71). New cstate: current_term: 1 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:35.561754 31475 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 20:46:35.562110 29295 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:35.566057 29163 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 20:46:35.568326 29557 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:35.568616 31868 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:35.570240 31735 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:35.583606 29033 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:35.594128 7878 tablet.cc:1722] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:35.594419 7878 tablet.cc:1620] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057: MemRowSet was empty: no flush needed.
I20240502 20:46:35.597699 7880 tablet.cc:1722] T e0bf2fff32214e349ae0cf765f7343e1 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:35.597980 7880 tablet.cc:1620] T e0bf2fff32214e349ae0cf765f7343e1 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:35.600860 29033 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=4b3014b5ed314186914b7365db776294] alter complete (version 2)
I20240502 20:46:35.601850 7881 tablet.cc:1722] T e0bf2fff32214e349ae0cf765f7343e1 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:35.602119 7881 tablet.cc:1620] T e0bf2fff32214e349ae0cf765f7343e1 P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:35.605391 29424 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 20:46:35.643143 7256 raft_consensus.cc:482] T da2da056404f4be5b9ed325bac7ddcb2 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:35.643260 7256 raft_consensus.cc:504] T da2da056404f4be5b9ed325bac7ddcb2 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:35.643589 7256 leader_election.cc:288] T da2da056404f4be5b9ed325bac7ddcb2 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:35.643916 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da2da056404f4be5b9ed325bac7ddcb2" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:35.644035 29313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da2da056404f4be5b9ed325bac7ddcb2" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:35.644241 29313 raft_consensus.cc:2379] T da2da056404f4be5b9ed325bac7ddcb2 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db17f26a73f3447b92aa9a947e3a7057 in current term 1: Already voted for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in this term.
I20240502 20:46:35.644477 29512 leader_election.cc:302] T da2da056404f4be5b9ed325bac7ddcb2 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057; no voters: 09258ac095664fa99caf0707262cde13, 2a9c1af8cf814030bde56b67ca5c2fbe
I20240502 20:46:35.644634 7256 raft_consensus.cc:3037] T da2da056404f4be5b9ed325bac7ddcb2 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:35.645677 7256 raft_consensus.cc:2726] T da2da056404f4be5b9ed325bac7ddcb2 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 20:46:35.660923 29313 consensus_queue.cc:237] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:35.662267 29574 raft_consensus.cc:1261] T d5ca50fc288f4957a2d6f04c23bd1491 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:35.662575 31755 raft_consensus.cc:1261] T d5ca50fc288f4957a2d6f04c23bd1491 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:35.662662 7910 consensus_queue.cc:1035] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:35.662966 7910 consensus_queue.cc:1035] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:35.664412 7181 raft_consensus.cc:2932] T d5ca50fc288f4957a2d6f04c23bd1491 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:35.664932 29574 raft_consensus.cc:2932] T d5ca50fc288f4957a2d6f04c23bd1491 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:35.665200 31755 raft_consensus.cc:2932] T d5ca50fc288f4957a2d6f04c23bd1491 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:35.668751 29034 catalog_manager.cc:5484] T d5ca50fc288f4957a2d6f04c23bd1491 P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: config changed from index 2 to 3, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) evicted. New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:35.672159 31868 tablet_service.cc:1514] Processing DeleteTablet for tablet d5ca50fc288f4957a2d6f04c23bd1491 with delete_type TABLET_DATA_TOMBSTONED (TS e6b60803d0764d228c863e2da65e3691 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:35.673771 7913 tablet_replica.cc:330] T d5ca50fc288f4957a2d6f04c23bd1491 P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:35.673926 7913 raft_consensus.cc:2229] T d5ca50fc288f4957a2d6f04c23bd1491 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:35.674065 7913 raft_consensus.cc:2258] T d5ca50fc288f4957a2d6f04c23bd1491 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:35.674747 7913 ts_tablet_manager.cc:1875] T d5ca50fc288f4957a2d6f04c23bd1491 P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:35.677064 7913 ts_tablet_manager.cc:1888] T d5ca50fc288f4957a2d6f04c23bd1491 P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 20:46:35.677137 7913 log.cc:1196] T d5ca50fc288f4957a2d6f04c23bd1491 P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/d5ca50fc288f4957a2d6f04c23bd1491
I20240502 20:46:35.677651 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet d5ca50fc288f4957a2d6f04c23bd1491 (table rebalancer_extra_table_28 [id=5dbd6a79afdc419e96698104b5e310ab]) successfully deleted
I20240502 20:46:35.703806 29035 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:35.711133 7878 tablet.cc:1722] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:35.713193 7881 tablet.cc:1722] T e0bf2fff32214e349ae0cf765f7343e1 P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:35.713536 7880 tablet.cc:1722] T e0bf2fff32214e349ae0cf765f7343e1 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:35.721979 6616 consensus_queue.cc:1035] T bcab2a7cb1974024899254facc6af322 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:35.722744 6616 consensus_queue.cc:1035] T bcab2a7cb1974024899254facc6af322 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:35.725943 29035 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=4b3014b5ed314186914b7365db776294] alter complete (version 3)
I20240502 20:46:35.766902 7022 consensus_queue.cc:1035] T da2da056404f4be5b9ed325bac7ddcb2 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:35.772857 7829 consensus_queue.cc:1035] T da2da056404f4be5b9ed325bac7ddcb2 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:35.799113 29163 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 20:46:35.801587 29295 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:35.805109 29424 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 20:46:35.807261 29557 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:35.809356 31475 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 20:46:35.817783 7940 raft_consensus.cc:482] T 90bab3d11d424b3abcdc74d5da51cbee P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:35.817891 7940 raft_consensus.cc:504] T 90bab3d11d424b3abcdc74d5da51cbee P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:35.818259 7940 leader_election.cc:288] T 90bab3d11d424b3abcdc74d5da51cbee P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
W20240502 20:46:35.819213 31691 leader_election.cc:334] T 90bab3d11d424b3abcdc74d5da51cbee P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:35.820024 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90bab3d11d424b3abcdc74d5da51cbee" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:35.820399 31692 leader_election.cc:302] T 90bab3d11d424b3abcdc74d5da51cbee P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7ab6b9053664ee5aa71c7978e26fe6a; no voters: 315915efcf2b413ca7f9d874338e1b2d, dbcdc6ac635b44b185c3eba63f66e9d6
I20240502 20:46:35.820590 7940 raft_consensus.cc:2726] T 90bab3d11d424b3abcdc74d5da51cbee P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20240502 20:46:35.825026 31691 consensus_peers.cc:462] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:35.830706 31868 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:35.831214 5346 raft_consensus.cc:482] T 03db15f1ebfb4e4ea271d6bb5c76110d P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:35.831302 5346 raft_consensus.cc:504] T 03db15f1ebfb4e4ea271d6bb5c76110d P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:35.831286 29033 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:35.831419 29033 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:35.831637 5346 leader_election.cc:288] T 03db15f1ebfb4e4ea271d6bb5c76110d P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:35.833765 29033 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet e0bf2fff32214e349ae0cf765f7343e1
I20240502 20:46:35.834589 31868 tablet_service.cc:1514] Processing DeleteTablet for tablet e0bf2fff32214e349ae0cf765f7343e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:35 UTC) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:35.834754 7913 tablet_replica.cc:330] T e0bf2fff32214e349ae0cf765f7343e1 P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:35.834883 7913 raft_consensus.cc:2229] T e0bf2fff32214e349ae0cf765f7343e1 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:35.834973 31735 tablet_service.cc:1514] Processing DeleteTablet for tablet e0bf2fff32214e349ae0cf765f7343e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:35 UTC) from {username='slave'} at 127.0.0.1:33864
I20240502 20:46:35.835001 7913 raft_consensus.cc:2258] T e0bf2fff32214e349ae0cf765f7343e1 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:35.835851 7913 ts_tablet_manager.cc:1875] T e0bf2fff32214e349ae0cf765f7343e1 P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:35.837936 7913 ts_tablet_manager.cc:1888] T e0bf2fff32214e349ae0cf765f7343e1 P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:35.837994 7913 log.cc:1196] T e0bf2fff32214e349ae0cf765f7343e1 P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/e0bf2fff32214e349ae0cf765f7343e1
I20240502 20:46:35.838213 7913 ts_tablet_manager.cc:1909] T e0bf2fff32214e349ae0cf765f7343e1 P e6b60803d0764d228c863e2da65e3691: Deleting consensus metadata
I20240502 20:46:35.839324 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet e0bf2fff32214e349ae0cf765f7343e1 (table rebalancer_dynamic_table_new_name [id=4b3014b5ed314186914b7365db776294]) successfully deleted
I20240502 20:46:35.839623 29557 tablet_service.cc:1514] Processing DeleteTablet for tablet e0bf2fff32214e349ae0cf765f7343e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:35 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:35.839668 31886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03db15f1ebfb4e4ea271d6bb5c76110d" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:35.843055 31735 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:35.843053 7910 raft_consensus.cc:482] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:35.843092 7981 tablet_replica.cc:330] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:35.843143 7910 raft_consensus.cc:504] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:35.843463 7910 leader_election.cc:288] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:35.843556 6850 raft_consensus.cc:482] T 0f5e277eec15411c8c1fd76189893ee9 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:35.843626 6850 raft_consensus.cc:504] T 0f5e277eec15411c8c1fd76189893ee9 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:35.843907 6850 leader_election.cc:288] T 0f5e277eec15411c8c1fd76189893ee9 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:35.844770 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe41756b5dc74f7f9105e002b8bc9c84" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:35.844879 29175 raft_consensus.cc:2454] T fe41756b5dc74f7f9105e002b8bc9c84 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:35.845574 29251 leader_election.cc:302] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:35.845886 7239 raft_consensus.cc:482] T fe41756b5dc74f7f9105e002b8bc9c84 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:35.845985 7239 raft_consensus.cc:504] T fe41756b5dc74f7f9105e002b8bc9c84 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:35.846279 7239 leader_election.cc:288] T fe41756b5dc74f7f9105e002b8bc9c84 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:35.846408 7910 raft_consensus.cc:2781] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 20:46:35.846493 7910 raft_consensus.cc:482] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:35.846531 7910 raft_consensus.cc:3037] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:35.847462 7910 raft_consensus.cc:504] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:35.847739 7910 leader_election.cc:288] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 2 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:35.848160 29313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe41756b5dc74f7f9105e002b8bc9c84" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:35.848276 29313 raft_consensus.cc:2379] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7796ea3fba294d17a34386873969431d in current term 2: Already voted for candidate 09258ac095664fa99caf0707262cde13 in this term.
I20240502 20:46:35.848718 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe41756b5dc74f7f9105e002b8bc9c84" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:35.848812 29175 raft_consensus.cc:3037] T fe41756b5dc74f7f9105e002b8bc9c84 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:35.849670 29175 raft_consensus.cc:2454] T fe41756b5dc74f7f9105e002b8bc9c84 P 7796ea3fba294d17a34386873969431d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 2.
I20240502 20:46:35.850245 29251 leader_election.cc:302] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
W20240502 20:46:35.850473 29249 leader_election.cc:334] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
W20240502 20:46:35.850775 29118 leader_election.cc:334] T fe41756b5dc74f7f9105e002b8bc9c84 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:35.850842 29118 leader_election.cc:302] T fe41756b5dc74f7f9105e002b8bc9c84 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7796ea3fba294d17a34386873969431d; no voters: 09258ac095664fa99caf0707262cde13, 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:35.851342 7910 raft_consensus.cc:2781] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:35.851429 7910 raft_consensus.cc:686] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13 [term 2 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:35.851543 7910 consensus_queue.cc:237] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:35.851647 7239 raft_consensus.cc:2726] T fe41756b5dc74f7f9105e002b8bc9c84 P 7796ea3fba294d17a34386873969431d [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20240502 20:46:35.852137 31822 leader_election.cc:334] T 0f5e277eec15411c8c1fd76189893ee9 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
W20240502 20:46:35.852298 29249 leader_election.cc:334] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
W20240502 20:46:35.853081 29511 leader_election.cc:334] T 03db15f1ebfb4e4ea271d6bb5c76110d P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:35.853152 29511 leader_election.cc:302] T 03db15f1ebfb4e4ea271d6bb5c76110d P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057; no voters: 315915efcf2b413ca7f9d874338e1b2d, e6b60803d0764d228c863e2da65e3691
I20240502 20:46:35.853332 5346 raft_consensus.cc:2726] T 03db15f1ebfb4e4ea271d6bb5c76110d P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 20:46:35.853739 29033 catalog_manager.cc:5484] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 1 to 2, leader changed from 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 2 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:35.854640 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f5e277eec15411c8c1fd76189893ee9" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:35.854997 31821 leader_election.cc:302] T 0f5e277eec15411c8c1fd76189893ee9 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e6b60803d0764d228c863e2da65e3691; no voters: 315915efcf2b413ca7f9d874338e1b2d, e7ab6b9053664ee5aa71c7978e26fe6a
I20240502 20:46:35.855219 6850 raft_consensus.cc:2726] T 0f5e277eec15411c8c1fd76189893ee9 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 20:46:35.855419 7981 raft_consensus.cc:2229] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:35.855651 7981 raft_consensus.cc:2258] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:35.857921 7981 ts_tablet_manager.cc:1875] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:35.860558 7981 ts_tablet_manager.cc:1888] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:35.860682 7981 log.cc:1196] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/e0bf2fff32214e349ae0cf765f7343e1
I20240502 20:46:35.861028 7981 ts_tablet_manager.cc:1909] T e0bf2fff32214e349ae0cf765f7343e1 P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:35.861851 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet e0bf2fff32214e349ae0cf765f7343e1 (table rebalancer_dynamic_table_new_name [id=4b3014b5ed314186914b7365db776294]) successfully deleted
I20240502 20:46:35.865196 7980 tablet_replica.cc:330] T e0bf2fff32214e349ae0cf765f7343e1 P e7ab6b9053664ee5aa71c7978e26fe6a: stopping tablet replica
I20240502 20:46:35.865329 7980 raft_consensus.cc:2229] T e0bf2fff32214e349ae0cf765f7343e1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:35.865438 7980 raft_consensus.cc:2258] T e0bf2fff32214e349ae0cf765f7343e1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:35.865955 7980 ts_tablet_manager.cc:1875] T e0bf2fff32214e349ae0cf765f7343e1 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:35.867812 7980 ts_tablet_manager.cc:1888] T e0bf2fff32214e349ae0cf765f7343e1 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:35.867923 7980 log.cc:1196] T e0bf2fff32214e349ae0cf765f7343e1 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/wals/e0bf2fff32214e349ae0cf765f7343e1
I20240502 20:46:35.868196 7980 ts_tablet_manager.cc:1909] T e0bf2fff32214e349ae0cf765f7343e1 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting consensus metadata
I20240502 20:46:35.868803 29026 catalog_manager.cc:4830] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): tablet e0bf2fff32214e349ae0cf765f7343e1 (table rebalancer_dynamic_table_new_name [id=4b3014b5ed314186914b7365db776294]) successfully deleted
I20240502 20:46:35.881716 7022 raft_consensus.cc:482] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:35.881814 7022 raft_consensus.cc:504] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:35.882169 7022 leader_election.cc:288] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:35.882485 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5993321dc8f54f14a89fa2ba4a44ccc7" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:35.882611 29444 raft_consensus.cc:2454] T 5993321dc8f54f14a89fa2ba4a44ccc7 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:35.882974 31430 leader_election.cc:302] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:35.883519 7022 raft_consensus.cc:2781] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 20:46:35.883584 7022 raft_consensus.cc:482] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting leader election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:35.883631 7022 raft_consensus.cc:3037] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:35.884470 7022 raft_consensus.cc:504] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:35.884770 7022 leader_election.cc:288] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 2 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:35.885221 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5993321dc8f54f14a89fa2ba4a44ccc7" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:35.885326 29444 raft_consensus.cc:3037] T 5993321dc8f54f14a89fa2ba4a44ccc7 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:35.886301 29444 raft_consensus.cc:2454] T 5993321dc8f54f14a89fa2ba4a44ccc7 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 2.
W20240502 20:46:35.886515 31429 leader_election.cc:334] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
W20240502 20:46:35.886634 31429 leader_election.cc:334] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:35.886713 31430 leader_election.cc:302] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, dbcdc6ac635b44b185c3eba63f66e9d6; no voters: 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:35.887028 7829 raft_consensus.cc:2781] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:35.887111 7829 raft_consensus.cc:686] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 2 LEADER]: Becoming Leader. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Running, Role: LEADER
I20240502 20:46:35.887241 7829 consensus_queue.cc:237] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:35.890395 29033 catalog_manager.cc:5484] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: term changed from 1 to 2, leader changed from 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) to 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69). New cstate: current_term: 2 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:35.936038 29041 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:35.944468 7940 consensus_queue.cc:1035] T 1202e6d31c4a4f0895b40002ac1d3dad P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:35.947604 31868 tablet_service.cc:1467] Processing CreateTablet for tablet 1e16a4014cf24e84abc7a7d42460dea3 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=089f58cd5abe45a7bcc326ddd6802ab3]), partition=
I20240502 20:46:35.947839 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e16a4014cf24e84abc7a7d42460dea3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:35.950114 29163 tablet_service.cc:1467] Processing CreateTablet for tablet 1e16a4014cf24e84abc7a7d42460dea3 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=089f58cd5abe45a7bcc326ddd6802ab3]), partition=
I20240502 20:46:35.950309 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e16a4014cf24e84abc7a7d42460dea3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:35.952447 8028 tablet_bootstrap.cc:492] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:35.953337 8028 tablet_bootstrap.cc:654] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:35.953565 8027 tablet_bootstrap.cc:492] T 1e16a4014cf24e84abc7a7d42460dea3 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:35.954174 7940 consensus_queue.cc:1035] T 1202e6d31c4a4f0895b40002ac1d3dad P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:35.955099 8027 tablet_bootstrap.cc:654] T 1e16a4014cf24e84abc7a7d42460dea3 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:35.956748 7940 raft_consensus.cc:482] T b15554a988034cc7ab18ac2330f1a356 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:35.956831 7940 raft_consensus.cc:504] T b15554a988034cc7ab18ac2330f1a356 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:35.957188 7940 leader_election.cc:288] T b15554a988034cc7ab18ac2330f1a356 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:35.958156 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15554a988034cc7ab18ac2330f1a356" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
W20240502 20:46:35.959017 31691 leader_election.cc:334] T b15554a988034cc7ab18ac2330f1a356 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:35.959079 31691 leader_election.cc:302] T b15554a988034cc7ab18ac2330f1a356 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7ab6b9053664ee5aa71c7978e26fe6a; no voters: 315915efcf2b413ca7f9d874338e1b2d, db17f26a73f3447b92aa9a947e3a7057
I20240502 20:46:35.959532 7940 raft_consensus.cc:2726] T b15554a988034cc7ab18ac2330f1a356 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20240502 20:46:35.961553 29027 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
W20240502 20:46:35.962360 29027 catalog_manager.cc:4573] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): CreateTablet RPC failed for tablet 1e16a4014cf24e84abc7a7d42460dea3: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:35.964321 8027 tablet_bootstrap.cc:492] T 1e16a4014cf24e84abc7a7d42460dea3 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:35.964464 8027 ts_tablet_manager.cc:1367] T 1e16a4014cf24e84abc7a7d42460dea3 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 20:46:35.965013 8027 raft_consensus.cc:348] T 1e16a4014cf24e84abc7a7d42460dea3 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:35.965198 8027 raft_consensus.cc:374] T 1e16a4014cf24e84abc7a7d42460dea3 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:35.965286 8027 raft_consensus.cc:729] T 1e16a4014cf24e84abc7a7d42460dea3 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:35.965461 8027 consensus_queue.cc:260] T 1e16a4014cf24e84abc7a7d42460dea3 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:35.965890 8027 ts_tablet_manager.cc:1398] T 1e16a4014cf24e84abc7a7d42460dea3 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:35.971051 8028 tablet_bootstrap.cc:492] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:35.971132 8028 ts_tablet_manager.cc:1367] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20240502 20:46:35.971570 8028 raft_consensus.cc:348] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:35.971669 8028 raft_consensus.cc:374] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:35.971702 8028 raft_consensus.cc:729] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:35.971807 8028 consensus_queue.cc:260] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:35.972096 8028 ts_tablet_manager.cc:1398] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:35.980492 6850 raft_consensus.cc:482] T 6e80723ee7624d698e9e0a0bd494e1f0 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:35.980574 6850 raft_consensus.cc:504] T 6e80723ee7624d698e9e0a0bd494e1f0 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:35.980873 6850 leader_election.cc:288] T 6e80723ee7624d698e9e0a0bd494e1f0 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:35.982738 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e80723ee7624d698e9e0a0bd494e1f0" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
W20240502 20:46:35.986594 31822 leader_election.cc:334] T 6e80723ee7624d698e9e0a0bd494e1f0 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:35.986670 31822 leader_election.cc:302] T 6e80723ee7624d698e9e0a0bd494e1f0 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e6b60803d0764d228c863e2da65e3691; no voters: 2a9c1af8cf814030bde56b67ca5c2fbe, 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:35.986821 6850 raft_consensus.cc:2726] T 6e80723ee7624d698e9e0a0bd494e1f0 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 20:46:36.004896 7940 raft_consensus.cc:482] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:36.004985 7940 raft_consensus.cc:504] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:36.005338 7940 leader_election.cc:288] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:36.005707 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5bdf4f57d2c4c828b94c7cc2e3c2847" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
W20240502 20:46:36.007357 31691 leader_election.cc:334] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.007427 31691 leader_election.cc:302] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7ab6b9053664ee5aa71c7978e26fe6a; no voters: 315915efcf2b413ca7f9d874338e1b2d, 7796ea3fba294d17a34386873969431d
I20240502 20:46:36.007637 7940 raft_consensus.cc:2726] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 20:46:36.025866 7979 raft_consensus.cc:482] T 358e4c761223404fbf0f467f38ce9b25 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:36.025975 7979 raft_consensus.cc:504] T 358e4c761223404fbf0f467f38ce9b25 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:36.026433 7979 leader_election.cc:288] T 358e4c761223404fbf0f467f38ce9b25 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:36.027269 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "358e4c761223404fbf0f467f38ce9b25" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:36.027281 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "358e4c761223404fbf0f467f38ce9b25" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:36.027734 29119 leader_election.cc:302] T 358e4c761223404fbf0f467f38ce9b25 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7796ea3fba294d17a34386873969431d; no voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e7ab6b9053664ee5aa71c7978e26fe6a
W20240502 20:46:36.027935 29118 leader_election.cc:334] T 358e4c761223404fbf0f467f38ce9b25 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.028288 7979 raft_consensus.cc:2726] T 358e4c761223404fbf0f467f38ce9b25 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 20:46:36.032969 7116 raft_consensus.cc:482] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:36.033071 7116 raft_consensus.cc:504] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:36.033432 7116 leader_election.cc:288] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:36.033658 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90bab3d11d424b3abcdc74d5da51cbee" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:36.033769 31757 raft_consensus.cc:2454] T 90bab3d11d424b3abcdc74d5da51cbee P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:36.034052 29379 leader_election.cc:302] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dbcdc6ac635b44b185c3eba63f66e9d6, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:36.034277 7116 raft_consensus.cc:2781] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 20:46:36.034325 7116 raft_consensus.cc:482] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:36.034360 7116 raft_consensus.cc:3037] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:36.034929 29295 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:36.035460 7116 raft_consensus.cc:504] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:36.035533 7940 raft_consensus.cc:482] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:36.035640 7940 raft_consensus.cc:504] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:36.035768 7116 leader_election.cc:288] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 2 election: Requested vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:36.035990 7940 leader_election.cc:288] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:36.036481 31886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f5e277eec15411c8c1fd76189893ee9" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:36.036612 31886 raft_consensus.cc:2454] T 0f5e277eec15411c8c1fd76189893ee9 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
W20240502 20:46:36.036885 29380 leader_election.cc:334] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.036931 31691 leader_election.cc:302] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e6b60803d0764d228c863e2da65e3691, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
W20240502 20:46:36.037460 29380 leader_election.cc:334] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.037448 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90bab3d11d424b3abcdc74d5da51cbee" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:36.037533 31757 raft_consensus.cc:3037] T 90bab3d11d424b3abcdc74d5da51cbee P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:36.038489 31757 raft_consensus.cc:2454] T 90bab3d11d424b3abcdc74d5da51cbee P e7ab6b9053664ee5aa71c7978e26fe6a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 2.
I20240502 20:46:36.038796 7940 raft_consensus.cc:2781] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 20:46:36.038862 7940 raft_consensus.cc:482] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:36.038897 7940 raft_consensus.cc:3037] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:36.039744 7940 raft_consensus.cc:504] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:36.040021 7940 leader_election.cc:288] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 2 election: Requested vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
W20240502 20:46:36.040539 31691 leader_election.cc:334] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.041087 31886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f5e277eec15411c8c1fd76189893ee9" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:36.041198 31886 raft_consensus.cc:3037] T 0f5e277eec15411c8c1fd76189893ee9 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:36.042151 31886 raft_consensus.cc:2454] T 0f5e277eec15411c8c1fd76189893ee9 P e6b60803d0764d228c863e2da65e3691 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 2.
I20240502 20:46:36.042520 29379 leader_election.cc:302] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: dbcdc6ac635b44b185c3eba63f66e9d6, e7ab6b9053664ee5aa71c7978e26fe6a; no voters: 315915efcf2b413ca7f9d874338e1b2d
W20240502 20:46:36.042770 31691 leader_election.cc:334] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.042917 31691 leader_election.cc:302] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e6b60803d0764d228c863e2da65e3691, e7ab6b9053664ee5aa71c7978e26fe6a; no voters: 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:36.043354 7940 raft_consensus.cc:2781] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:36.043438 7940 raft_consensus.cc:686] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 2 LEADER]: Becoming Leader. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Running, Role: LEADER
I20240502 20:46:36.043553 7940 consensus_queue.cc:237] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:36.043994 31475 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 20:46:36.046255 29041 catalog_manager.cc:5484] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: term changed from 1 to 2, leader changed from 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) to e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71). New cstate: current_term: 2 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:36.048306 29163 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 20:46:36.050783 29035 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_60"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:36.051883 7116 raft_consensus.cc:2781] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:36.051965 7116 raft_consensus.cc:686] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6 [term 2 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:36.052093 7116 consensus_queue.cc:237] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:36.056576 5346 raft_consensus.cc:482] T 08feee781db04883b6dcc030bdaa8afe P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:36.056670 5346 raft_consensus.cc:504] T 08feee781db04883b6dcc030bdaa8afe P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:36.057029 5346 leader_election.cc:288] T 08feee781db04883b6dcc030bdaa8afe P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:36.057608 29041 catalog_manager.cc:5484] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 1 to 2, leader changed from 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 2 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:36.058359 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08feee781db04883b6dcc030bdaa8afe" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:36.059010 5346 raft_consensus.cc:482] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:36.059093 5346 raft_consensus.cc:504] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:36.059414 5346 leader_election.cc:288] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
W20240502 20:46:36.059557 29511 leader_election.cc:334] T 08feee781db04883b6dcc030bdaa8afe P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.059619 29511 leader_election.cc:302] T 08feee781db04883b6dcc030bdaa8afe P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057; no voters: 2a9c1af8cf814030bde56b67ca5c2fbe, 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:36.059979 5346 raft_consensus.cc:2726] T 08feee781db04883b6dcc030bdaa8afe P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 20:46:36.060077 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15554a988034cc7ab18ac2330f1a356" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:36.060190 31757 raft_consensus.cc:2454] T b15554a988034cc7ab18ac2330f1a356 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
W20240502 20:46:36.060355 29511 leader_election.cc:334] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.060618 29510 leader_election.cc:302] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, e7ab6b9053664ee5aa71c7978e26fe6a; no voters: 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:36.060894 5346 raft_consensus.cc:2781] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 20:46:36.060962 5346 raft_consensus.cc:482] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:36.061005 5346 raft_consensus.cc:3037] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:36.062069 5346 raft_consensus.cc:504] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:36.062386 5346 leader_election.cc:288] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 2 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:36.062811 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15554a988034cc7ab18ac2330f1a356" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:36.062914 31755 raft_consensus.cc:3037] T b15554a988034cc7ab18ac2330f1a356 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:36.063846 31755 raft_consensus.cc:2454] T b15554a988034cc7ab18ac2330f1a356 P e7ab6b9053664ee5aa71c7978e26fe6a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 2.
I20240502 20:46:36.064996 29425 tablet_service.cc:1467] Processing CreateTablet for tablet 40305eb373a542f4a6aba1a55947442b (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=4f9870837fe045908f048cbe22142c76]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:36.065235 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40305eb373a542f4a6aba1a55947442b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:36.065727 29295 tablet_service.cc:1467] Processing CreateTablet for tablet ab91f7c07fd14fbb8cd8a70bf8f616ab (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=4f9870837fe045908f048cbe22142c76]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:36.065958 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab91f7c07fd14fbb8cd8a70bf8f616ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:36.066704 29424 tablet_service.cc:1467] Processing CreateTablet for tablet 22571ccf8d3c4db98af98d75da3cb954 (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=4f9870837fe045908f048cbe22142c76]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:36.066916 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22571ccf8d3c4db98af98d75da3cb954. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:36.067430 8060 tablet_bootstrap.cc:492] T 40305eb373a542f4a6aba1a55947442b P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:36.067809 29163 tablet_service.cc:1467] Processing CreateTablet for tablet 22571ccf8d3c4db98af98d75da3cb954 (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=4f9870837fe045908f048cbe22142c76]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:36.068063 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22571ccf8d3c4db98af98d75da3cb954. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:36.068709 8060 tablet_bootstrap.cc:654] T 40305eb373a542f4a6aba1a55947442b P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:36.070227 8028 tablet_bootstrap.cc:492] T 22571ccf8d3c4db98af98d75da3cb954 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:36.070309 31735 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
W20240502 20:46:36.070362 29511 leader_election.cc:334] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.071339 8028 tablet_bootstrap.cc:654] T 22571ccf8d3c4db98af98d75da3cb954 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:36.071885 29510 leader_election.cc:302] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, e7ab6b9053664ee5aa71c7978e26fe6a; no voters: 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:36.072917 7940 raft_consensus.cc:482] T 58d4026c208f45d8870a2eaa1c97efab P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:36.072994 7940 raft_consensus.cc:504] T 58d4026c208f45d8870a2eaa1c97efab P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:36.073300 7940 leader_election.cc:288] T 58d4026c208f45d8870a2eaa1c97efab P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:36.074074 5346 raft_consensus.cc:2781] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:36.074204 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58d4026c208f45d8870a2eaa1c97efab" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:36.074344 5346 raft_consensus.cc:686] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057 [term 2 LEADER]: Becoming Leader. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Running, Role: LEADER
I20240502 20:46:36.074492 5346 consensus_queue.cc:237] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:36.074851 31868 tablet_service.cc:1467] Processing CreateTablet for tablet 22571ccf8d3c4db98af98d75da3cb954 (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=4f9870837fe045908f048cbe22142c76]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:36.075095 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22571ccf8d3c4db98af98d75da3cb954. 1 dirs total, 0 dirs full, 0 dirs failed
W20240502 20:46:36.075501 31691 leader_election.cc:334] T 58d4026c208f45d8870a2eaa1c97efab P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.075862 29557 tablet_service.cc:1467] Processing CreateTablet for tablet ab91f7c07fd14fbb8cd8a70bf8f616ab (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=4f9870837fe045908f048cbe22142c76]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:36.076129 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab91f7c07fd14fbb8cd8a70bf8f616ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:36.077136 8027 tablet_bootstrap.cc:492] T 22571ccf8d3c4db98af98d75da3cb954 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:36.078081 8027 tablet_bootstrap.cc:654] T 22571ccf8d3c4db98af98d75da3cb954 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:36.078279 8063 tablet_bootstrap.cc:492] T ab91f7c07fd14fbb8cd8a70bf8f616ab P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:36.078480 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 40305eb373a542f4a6aba1a55947442b (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=4f9870837fe045908f048cbe22142c76]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:36.078687 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40305eb373a542f4a6aba1a55947442b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:36.079295 8063 tablet_bootstrap.cc:654] T ab91f7c07fd14fbb8cd8a70bf8f616ab P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:36.080653 31691 leader_election.cc:302] T 58d4026c208f45d8870a2eaa1c97efab P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7ab6b9053664ee5aa71c7978e26fe6a; no voters: 315915efcf2b413ca7f9d874338e1b2d, db17f26a73f3447b92aa9a947e3a7057
I20240502 20:46:36.081158 7940 raft_consensus.cc:2726] T 58d4026c208f45d8870a2eaa1c97efab P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 20:46:36.081509 8065 tablet_bootstrap.cc:492] T ab91f7c07fd14fbb8cd8a70bf8f616ab P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:36.082376 8065 tablet_bootstrap.cc:654] T ab91f7c07fd14fbb8cd8a70bf8f616ab P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:36.083318 29041 catalog_manager.cc:5484] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: term changed from 1 to 2, leader changed from 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) to db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68). New cstate: current_term: 2 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:36.083810 8060 tablet_bootstrap.cc:492] T 40305eb373a542f4a6aba1a55947442b P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:36.083890 8060 ts_tablet_manager.cc:1367] T 40305eb373a542f4a6aba1a55947442b P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.000s
I20240502 20:46:36.084244 8060 raft_consensus.cc:348] T 40305eb373a542f4a6aba1a55947442b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:36.084328 8060 raft_consensus.cc:374] T 40305eb373a542f4a6aba1a55947442b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:36.084329 8027 tablet_bootstrap.cc:492] T 22571ccf8d3c4db98af98d75da3cb954 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:36.084354 8060 raft_consensus.cc:729] T 40305eb373a542f4a6aba1a55947442b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:36.084404 8027 ts_tablet_manager.cc:1367] T 22571ccf8d3c4db98af98d75da3cb954 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 20:46:36.084463 8060 consensus_queue.cc:260] T 40305eb373a542f4a6aba1a55947442b P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:36.084794 8027 raft_consensus.cc:348] T 22571ccf8d3c4db98af98d75da3cb954 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:36.084929 8027 raft_consensus.cc:374] T 22571ccf8d3c4db98af98d75da3cb954 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:36.084970 8027 raft_consensus.cc:729] T 22571ccf8d3c4db98af98d75da3cb954 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:36.085084 8027 consensus_queue.cc:260] T 22571ccf8d3c4db98af98d75da3cb954 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:36.085398 8027 ts_tablet_manager.cc:1398] T 22571ccf8d3c4db98af98d75da3cb954 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:36.085567 8027 tablet_bootstrap.cc:492] T 40305eb373a542f4a6aba1a55947442b P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:36.085606 8028 tablet_bootstrap.cc:492] T 22571ccf8d3c4db98af98d75da3cb954 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:36.085680 8028 ts_tablet_manager.cc:1367] T 22571ccf8d3c4db98af98d75da3cb954 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 20:46:36.085857 8060 ts_tablet_manager.cc:1398] T 40305eb373a542f4a6aba1a55947442b P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:36.086006 8060 tablet_bootstrap.cc:492] T 22571ccf8d3c4db98af98d75da3cb954 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:36.086087 8028 raft_consensus.cc:348] T 22571ccf8d3c4db98af98d75da3cb954 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:36.086180 8028 raft_consensus.cc:374] T 22571ccf8d3c4db98af98d75da3cb954 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:36.086210 8028 raft_consensus.cc:729] T 22571ccf8d3c4db98af98d75da3cb954 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:36.086387 8027 tablet_bootstrap.cc:654] T 40305eb373a542f4a6aba1a55947442b P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:36.086338 8028 consensus_queue.cc:260] T 22571ccf8d3c4db98af98d75da3cb954 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:36.086658 8028 ts_tablet_manager.cc:1398] T 22571ccf8d3c4db98af98d75da3cb954 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:36.087276 8060 tablet_bootstrap.cc:654] T 22571ccf8d3c4db98af98d75da3cb954 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:36.088081 8065 tablet_bootstrap.cc:492] T ab91f7c07fd14fbb8cd8a70bf8f616ab P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:36.088156 8065 ts_tablet_manager.cc:1367] T ab91f7c07fd14fbb8cd8a70bf8f616ab P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20240502 20:46:36.088361 8060 tablet_bootstrap.cc:492] T 22571ccf8d3c4db98af98d75da3cb954 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:36.088435 8060 ts_tablet_manager.cc:1367] T 22571ccf8d3c4db98af98d75da3cb954 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:36.088596 8065 raft_consensus.cc:348] T ab91f7c07fd14fbb8cd8a70bf8f616ab P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:36.088688 8065 raft_consensus.cc:374] T ab91f7c07fd14fbb8cd8a70bf8f616ab P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:36.088719 8065 raft_consensus.cc:729] T ab91f7c07fd14fbb8cd8a70bf8f616ab P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:36.088825 8060 raft_consensus.cc:348] T 22571ccf8d3c4db98af98d75da3cb954 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:36.088829 8065 consensus_queue.cc:260] T ab91f7c07fd14fbb8cd8a70bf8f616ab P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:36.088917 8060 raft_consensus.cc:374] T 22571ccf8d3c4db98af98d75da3cb954 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:36.088948 8060 raft_consensus.cc:729] T 22571ccf8d3c4db98af98d75da3cb954 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:36.089052 8060 consensus_queue.cc:260] T 22571ccf8d3c4db98af98d75da3cb954 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:36.089107 8065 ts_tablet_manager.cc:1398] T ab91f7c07fd14fbb8cd8a70bf8f616ab P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:36.089304 8060 ts_tablet_manager.cc:1398] T 22571ccf8d3c4db98af98d75da3cb954 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:36.091642 8063 tablet_bootstrap.cc:492] T ab91f7c07fd14fbb8cd8a70bf8f616ab P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:36.091795 8063 ts_tablet_manager.cc:1367] T ab91f7c07fd14fbb8cd8a70bf8f616ab P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 20:46:36.092307 8063 raft_consensus.cc:348] T ab91f7c07fd14fbb8cd8a70bf8f616ab P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:36.092481 8063 raft_consensus.cc:374] T ab91f7c07fd14fbb8cd8a70bf8f616ab P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:36.092561 8063 raft_consensus.cc:729] T ab91f7c07fd14fbb8cd8a70bf8f616ab P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:36.092696 8063 consensus_queue.cc:260] T ab91f7c07fd14fbb8cd8a70bf8f616ab P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:36.093149 8063 ts_tablet_manager.cc:1398] T ab91f7c07fd14fbb8cd8a70bf8f616ab P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:36.095028 31867 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:36.095774 29557 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:36.106503 29424 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 20:46:36.121767 5346 raft_consensus.cc:482] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:36.121944 5346 raft_consensus.cc:504] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:36.123049 5346 leader_election.cc:288] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
W20240502 20:46:36.123241 29511 leader_election.cc:334] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.127673 6234 raft_consensus.cc:482] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:36.127810 8027 tablet_bootstrap.cc:492] T 40305eb373a542f4a6aba1a55947442b P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:36.127772 6234 raft_consensus.cc:504] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:36.127900 8027 ts_tablet_manager.cc:1367] T 40305eb373a542f4a6aba1a55947442b P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.042s user 0.000s sys 0.001s
I20240502 20:46:36.128113 6234 leader_election.cc:288] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:36.128839 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "378b1151c07648919504bacdcb20a087" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:36.128952 29575 raft_consensus.cc:2454] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:36.129052 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "378b1151c07648919504bacdcb20a087" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:36.129168 29444 raft_consensus.cc:2454] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
W20240502 20:46:36.129374 29380 leader_election.cc:334] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.129577 29380 leader_election.cc:302] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, dbcdc6ac635b44b185c3eba63f66e9d6; no voters: 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:36.129647 8027 raft_consensus.cc:348] T 40305eb373a542f4a6aba1a55947442b P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:36.129765 8027 raft_consensus.cc:374] T 40305eb373a542f4a6aba1a55947442b P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:36.129801 8027 raft_consensus.cc:729] T 40305eb373a542f4a6aba1a55947442b P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:36.129915 8027 consensus_queue.cc:260] T 40305eb373a542f4a6aba1a55947442b P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:36.130160 8027 ts_tablet_manager.cc:1398] T 40305eb373a542f4a6aba1a55947442b P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:36.131342 29512 leader_election.cc:302] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, dbcdc6ac635b44b185c3eba63f66e9d6; no voters: 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:36.131605 6234 raft_consensus.cc:2781] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 20:46:36.131656 6234 raft_consensus.cc:482] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:36.131690 6234 raft_consensus.cc:3037] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:36.132678 6234 raft_consensus.cc:504] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:36.132992 6234 leader_election.cc:288] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 2 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:36.133206 5346 raft_consensus.cc:2781] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 20:46:36.133255 5346 raft_consensus.cc:482] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:36.133287 5346 raft_consensus.cc:3037] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:36.134212 5346 raft_consensus.cc:504] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:36.134550 5346 leader_election.cc:288] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 2 election: Requested vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:36.135721 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "378b1151c07648919504bacdcb20a087" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:36.135844 29444 raft_consensus.cc:2379] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate db17f26a73f3447b92aa9a947e3a7057 in current term 2: Already voted for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in this term.
W20240502 20:46:36.136660 29380 leader_election.cc:334] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.137146 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "378b1151c07648919504bacdcb20a087" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:36.137260 29575 raft_consensus.cc:2379] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate dbcdc6ac635b44b185c3eba63f66e9d6 in current term 2: Already voted for candidate db17f26a73f3447b92aa9a947e3a7057 in this term.
I20240502 20:46:36.138142 29380 leader_election.cc:302] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbcdc6ac635b44b185c3eba63f66e9d6; no voters: 315915efcf2b413ca7f9d874338e1b2d, db17f26a73f3447b92aa9a947e3a7057
I20240502 20:46:36.138257 6850 raft_consensus.cc:482] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:36.138330 6850 raft_consensus.cc:504] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:36.138361 6234 raft_consensus.cc:2726] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20240502 20:46:36.138674 6850 leader_election.cc:288] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:36.139353 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03db15f1ebfb4e4ea271d6bb5c76110d" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:36.139457 29575 raft_consensus.cc:2454] T 03db15f1ebfb4e4ea271d6bb5c76110d P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:36.139889 31822 leader_election.cc:302] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:36.140561 6850 raft_consensus.cc:2781] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 20:46:36.140614 6850 raft_consensus.cc:482] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:36.140650 6850 raft_consensus.cc:3037] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:36.141634 6850 raft_consensus.cc:504] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:36.141937 6850 leader_election.cc:288] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 2 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
W20240502 20:46:36.142127 31822 leader_election.cc:334] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.142817 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03db15f1ebfb4e4ea271d6bb5c76110d" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:36.142900 29575 raft_consensus.cc:3037] T 03db15f1ebfb4e4ea271d6bb5c76110d P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:36.143745 29575 raft_consensus.cc:2454] T 03db15f1ebfb4e4ea271d6bb5c76110d P db17f26a73f3447b92aa9a947e3a7057 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 2.
I20240502 20:46:36.144205 31822 leader_election.cc:302] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, e6b60803d0764d228c863e2da65e3691; no voters:
W20240502 20:46:36.144727 29511 leader_election.cc:334] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.144804 29511 leader_election.cc:302] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057; no voters: 315915efcf2b413ca7f9d874338e1b2d, dbcdc6ac635b44b185c3eba63f66e9d6
W20240502 20:46:36.145068 31822 leader_election.cc:334] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.145258 6850 raft_consensus.cc:2781] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:36.145334 6850 raft_consensus.cc:686] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 [term 2 LEADER]: Becoming Leader. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Running, Role: LEADER
I20240502 20:46:36.145448 6850 consensus_queue.cc:237] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:36.146939 5346 raft_consensus.cc:2726] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20240502 20:46:36.147863 29041 catalog_manager.cc:5484] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 reported cstate change: term changed from 1 to 2, leader changed from 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) to e6b60803d0764d228c863e2da65e3691 (127.22.45.72). New cstate: current_term: 2 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:36.215549 7256 raft_consensus.cc:482] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:36.215657 7256 raft_consensus.cc:504] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:36.216041 7256 leader_election.cc:288] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
W20240502 20:46:36.216922 29511 leader_election.cc:334] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.217063 7940 raft_consensus.cc:482] T 358e4c761223404fbf0f467f38ce9b25 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:36.217075 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58d4026c208f45d8870a2eaa1c97efab" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:36.217168 7940 raft_consensus.cc:504] T 358e4c761223404fbf0f467f38ce9b25 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:36.217223 31755 raft_consensus.cc:2454] T 58d4026c208f45d8870a2eaa1c97efab P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:36.217489 29510 leader_election.cc:302] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, e7ab6b9053664ee5aa71c7978e26fe6a; no voters: 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:36.217602 7940 leader_election.cc:288] T 358e4c761223404fbf0f467f38ce9b25 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:36.217715 7256 raft_consensus.cc:2781] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 20:46:36.217765 7256 raft_consensus.cc:482] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:36.217803 7256 raft_consensus.cc:3037] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:36.217981 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "358e4c761223404fbf0f467f38ce9b25" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:36.218122 29175 raft_consensus.cc:2454] T 358e4c761223404fbf0f467f38ce9b25 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:36.218503 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "358e4c761223404fbf0f467f38ce9b25" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:36.218942 7256 raft_consensus.cc:504] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:36.219260 7256 leader_election.cc:288] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 2 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:36.220539 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58d4026c208f45d8870a2eaa1c97efab" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:36.220638 31755 raft_consensus.cc:3037] T 58d4026c208f45d8870a2eaa1c97efab P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:36.221689 31755 raft_consensus.cc:2454] T 58d4026c208f45d8870a2eaa1c97efab P e7ab6b9053664ee5aa71c7978e26fe6a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 2.
I20240502 20:46:36.222326 29510 leader_election.cc:302] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:36.222497 7256 raft_consensus.cc:2781] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:36.222584 7256 raft_consensus.cc:686] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 [term 2 LEADER]: Becoming Leader. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Running, Role: LEADER
I20240502 20:46:36.222731 7256 consensus_queue.cc:237] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
W20240502 20:46:36.222780 31691 leader_election.cc:334] T 358e4c761223404fbf0f467f38ce9b25 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.222831 31691 leader_election.cc:302] T 358e4c761223404fbf0f467f38ce9b25 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters: 2a9c1af8cf814030bde56b67ca5c2fbe, 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:36.222947 7940 raft_consensus.cc:2726] T 358e4c761223404fbf0f467f38ce9b25 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20240502 20:46:36.223912 29511 leader_election.cc:334] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.225469 29035 catalog_manager.cc:5484] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: term changed from 1 to 2, leader changed from 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) to db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68). New cstate: current_term: 2 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:36.227488 7022 raft_consensus.cc:482] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:36.227578 7022 raft_consensus.cc:504] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:36.228005 7022 leader_election.cc:288] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:36.228276 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "358e4c761223404fbf0f467f38ce9b25" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:36.228394 29175 raft_consensus.cc:2454] T 358e4c761223404fbf0f467f38ce9b25 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:36.229130 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "358e4c761223404fbf0f467f38ce9b25" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:36.229233 31755 raft_consensus.cc:2454] T 358e4c761223404fbf0f467f38ce9b25 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:36.229489 31428 leader_election.cc:302] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, 7796ea3fba294d17a34386873969431d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:36.229650 7829 raft_consensus.cc:2781] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 20:46:36.229714 7829 raft_consensus.cc:482] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting leader election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:36.229753 7829 raft_consensus.cc:3037] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:36.230741 7829 raft_consensus.cc:504] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:36.231081 7829 leader_election.cc:288] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 2 election: Requested vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:36.231462 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "358e4c761223404fbf0f467f38ce9b25" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:36.231572 29175 raft_consensus.cc:3037] T 358e4c761223404fbf0f467f38ce9b25 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:36.231637 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "358e4c761223404fbf0f467f38ce9b25" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:36.231729 31755 raft_consensus.cc:3037] T 358e4c761223404fbf0f467f38ce9b25 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Advancing to term 2
W20240502 20:46:36.232195 31429 leader_election.cc:334] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.232609 31755 raft_consensus.cc:2454] T 358e4c761223404fbf0f467f38ce9b25 P e7ab6b9053664ee5aa71c7978e26fe6a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 2.
I20240502 20:46:36.232689 29175 raft_consensus.cc:2454] T 358e4c761223404fbf0f467f38ce9b25 P 7796ea3fba294d17a34386873969431d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 2.
W20240502 20:46:36.234174 31429 leader_election.cc:334] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.236682 31431 leader_election.cc:302] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, 7796ea3fba294d17a34386873969431d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters: 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:36.237157 7829 raft_consensus.cc:2781] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:36.237250 7829 raft_consensus.cc:686] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 2 LEADER]: Becoming Leader. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Running, Role: LEADER
I20240502 20:46:36.237368 7829 consensus_queue.cc:237] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:36.243295 29035 catalog_manager.cc:5484] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: term changed from 1 to 2, leader changed from 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) to 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69). New cstate: current_term: 2 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:36.252039 7256 raft_consensus.cc:482] T ab91f7c07fd14fbb8cd8a70bf8f616ab P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:36.252123 7256 raft_consensus.cc:504] T ab91f7c07fd14fbb8cd8a70bf8f616ab P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:36.252460 7256 leader_election.cc:288] T ab91f7c07fd14fbb8cd8a70bf8f616ab P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:36.253283 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab91f7c07fd14fbb8cd8a70bf8f616ab" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:36.253413 29315 raft_consensus.cc:2454] T ab91f7c07fd14fbb8cd8a70bf8f616ab P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:36.253866 29512 leader_election.cc:302] T ab91f7c07fd14fbb8cd8a70bf8f616ab P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:36.254063 7256 raft_consensus.cc:2781] T ab91f7c07fd14fbb8cd8a70bf8f616ab P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:36.254114 7256 raft_consensus.cc:482] T ab91f7c07fd14fbb8cd8a70bf8f616ab P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:36.254149 7256 raft_consensus.cc:3037] T ab91f7c07fd14fbb8cd8a70bf8f616ab P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:36.255129 7256 raft_consensus.cc:504] T ab91f7c07fd14fbb8cd8a70bf8f616ab P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:36.255434 7256 leader_election.cc:288] T ab91f7c07fd14fbb8cd8a70bf8f616ab P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
W20240502 20:46:36.256094 29511 leader_election.cc:334] T ab91f7c07fd14fbb8cd8a70bf8f616ab P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.256759 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab91f7c07fd14fbb8cd8a70bf8f616ab" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:36.256875 29315 raft_consensus.cc:3037] T ab91f7c07fd14fbb8cd8a70bf8f616ab P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
W20240502 20:46:36.257130 29511 leader_election.cc:334] T ab91f7c07fd14fbb8cd8a70bf8f616ab P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.257690 29315 raft_consensus.cc:2454] T ab91f7c07fd14fbb8cd8a70bf8f616ab P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:36.257921 29512 leader_election.cc:302] T ab91f7c07fd14fbb8cd8a70bf8f616ab P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057; no voters: 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:36.258086 7256 raft_consensus.cc:2781] T ab91f7c07fd14fbb8cd8a70bf8f616ab P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:36.258180 7256 raft_consensus.cc:686] T ab91f7c07fd14fbb8cd8a70bf8f616ab P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Becoming Leader. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Running, Role: LEADER
I20240502 20:46:36.258306 7256 consensus_queue.cc:237] T ab91f7c07fd14fbb8cd8a70bf8f616ab P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:36.260655 29039 catalog_manager.cc:5484] T ab91f7c07fd14fbb8cd8a70bf8f616ab P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: term changed from 0 to 1, leader changed from <none> to db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
I20240502 20:46:36.286218 7829 raft_consensus.cc:482] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:36.286325 7829 raft_consensus.cc:504] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:36.286679 7829 leader_election.cc:288] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:36.287180 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08feee781db04883b6dcc030bdaa8afe" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:36.287305 29575 raft_consensus.cc:2454] T 08feee781db04883b6dcc030bdaa8afe P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:36.287590 31429 leader_election.cc:302] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:36.287663 7979 raft_consensus.cc:482] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:36.287758 7979 raft_consensus.cc:504] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:36.287926 7829 raft_consensus.cc:482] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:36.288007 7829 raft_consensus.cc:504] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
W20240502 20:46:36.288065 31429 leader_election.cc:334] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.288136 7979 leader_election.cc:288] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:36.288298 7829 leader_election.cc:288] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:36.288383 7829 raft_consensus.cc:2781] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 20:46:36.288417 7829 raft_consensus.cc:482] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting leader election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:36.288450 7829 raft_consensus.cc:3037] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:36.288599 31886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e16a4014cf24e84abc7a7d42460dea3" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:36.288710 31886 raft_consensus.cc:2454] T 1e16a4014cf24e84abc7a7d42460dea3 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:36.288762 31887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e80723ee7624d698e9e0a0bd494e1f0" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:36.288872 31887 raft_consensus.cc:2454] T 6e80723ee7624d698e9e0a0bd494e1f0 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:36.289186 31429 leader_election.cc:302] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:36.289316 6616 raft_consensus.cc:2781] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 20:46:36.289361 6616 raft_consensus.cc:482] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting leader election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:36.289397 6616 raft_consensus.cc:3037] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:36.289458 7829 raft_consensus.cc:504] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:36.289696 7829 leader_election.cc:288] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 2 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
W20240502 20:46:36.289978 31429 leader_election.cc:334] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.290369 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08feee781db04883b6dcc030bdaa8afe" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:36.290390 6616 raft_consensus.cc:504] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:36.290498 29575 raft_consensus.cc:3037] T 08feee781db04883b6dcc030bdaa8afe P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:36.290697 6616 leader_election.cc:288] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 2 election: Requested vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:36.290971 29118 leader_election.cc:302] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:36.291390 29575 raft_consensus.cc:2454] T 08feee781db04883b6dcc030bdaa8afe P db17f26a73f3447b92aa9a947e3a7057 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 2.
I20240502 20:46:36.291440 7239 raft_consensus.cc:2781] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:36.291493 7239 raft_consensus.cc:482] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:36.291472 31887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e80723ee7624d698e9e0a0bd494e1f0" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:36.291527 7239 raft_consensus.cc:3037] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:36.291560 31887 raft_consensus.cc:3037] T 6e80723ee7624d698e9e0a0bd494e1f0 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:36.292459 31887 raft_consensus.cc:2454] T 6e80723ee7624d698e9e0a0bd494e1f0 P e6b60803d0764d228c863e2da65e3691 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 2.
I20240502 20:46:36.292459 7239 raft_consensus.cc:504] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:36.292762 7239 leader_election.cc:288] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
W20240502 20:46:36.293272 31429 leader_election.cc:334] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
W20240502 20:46:36.293279 29118 leader_election.cc:334] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.293480 31429 leader_election.cc:302] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:36.293638 31429 leader_election.cc:302] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, db17f26a73f3447b92aa9a947e3a7057; no voters: 315915efcf2b413ca7f9d874338e1b2d
W20240502 20:46:36.293818 29118 leader_election.cc:334] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.293648 7022 raft_consensus.cc:2781] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:36.293920 7022 raft_consensus.cc:686] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 2 LEADER]: Becoming Leader. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Running, Role: LEADER
I20240502 20:46:36.294046 7022 consensus_queue.cc:237] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
W20240502 20:46:36.294294 31429 leader_election.cc:334] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.294668 7022 raft_consensus.cc:2781] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:36.294747 7022 raft_consensus.cc:686] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe [term 2 LEADER]: Becoming Leader. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Running, Role: LEADER
I20240502 20:46:36.294865 7022 consensus_queue.cc:237] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:36.295234 31887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e16a4014cf24e84abc7a7d42460dea3" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:36.295328 31887 raft_consensus.cc:3037] T 1e16a4014cf24e84abc7a7d42460dea3 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:36.296164 31887 raft_consensus.cc:2454] T 1e16a4014cf24e84abc7a7d42460dea3 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:36.296749 29118 leader_election.cc:302] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e6b60803d0764d228c863e2da65e3691; no voters: 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:36.296947 7239 raft_consensus.cc:2781] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:36.297016 7239 raft_consensus.cc:686] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:36.297134 7239 consensus_queue.cc:237] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:36.297528 29040 catalog_manager.cc:5484] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: term changed from 1 to 2, leader changed from 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) to 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69). New cstate: current_term: 2 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:36.299039 29036 catalog_manager.cc:5484] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:36.302179 29036 catalog_manager.cc:5484] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: term changed from 1 to 2, leader changed from 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) to 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69). New cstate: current_term: 2 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
W20240502 20:46:36.325127 31429 consensus_peers.cc:485] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:36.325388 7829 raft_consensus.cc:482] T 8d5cca35ba4b47748235d1463ec161d6 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:36.325474 7829 raft_consensus.cc:504] T 8d5cca35ba4b47748235d1463ec161d6 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:36.325781 7829 leader_election.cc:288] T 8d5cca35ba4b47748235d1463ec161d6 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:36.326058 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d5cca35ba4b47748235d1463ec161d6" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
W20240502 20:46:36.326867 31429 leader_election.cc:334] T 8d5cca35ba4b47748235d1463ec161d6 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.326936 31429 leader_election.cc:302] T 8d5cca35ba4b47748235d1463ec161d6 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe; no voters: 315915efcf2b413ca7f9d874338e1b2d, e7ab6b9053664ee5aa71c7978e26fe6a
I20240502 20:46:36.327069 7829 raft_consensus.cc:2726] T 8d5cca35ba4b47748235d1463ec161d6 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 20:46:36.333827 7940 raft_consensus.cc:482] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:36.333930 7940 raft_consensus.cc:504] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:36.334288 7940 leader_election.cc:288] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:36.335186 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d5cca35ba4b47748235d1463ec161d6" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:36.335263 7979 raft_consensus.cc:482] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:36.335299 31494 raft_consensus.cc:2454] T 8d5cca35ba4b47748235d1463ec161d6 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:36.335335 7979 raft_consensus.cc:504] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:36.335645 7979 leader_election.cc:288] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
W20240502 20:46:36.336020 31691 leader_election.cc:334] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.336561 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5bdf4f57d2c4c828b94c7cc2e3c2847" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
W20240502 20:46:36.336597 29118 leader_election.cc:334] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.336673 31755 raft_consensus.cc:2454] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:36.336704 31692 leader_election.cc:302] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e7ab6b9053664ee5aa71c7978e26fe6a; no voters: 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:36.336988 29117 leader_election.cc:302] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters: 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:36.337124 7979 raft_consensus.cc:2781] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 20:46:36.337173 7979 raft_consensus.cc:482] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:36.337198 7940 raft_consensus.cc:2781] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 20:46:36.337213 7979 raft_consensus.cc:3037] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:36.337241 7940 raft_consensus.cc:482] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 315915efcf2b413ca7f9d874338e1b2d)
I20240502 20:46:36.337276 7940 raft_consensus.cc:3037] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:36.338163 7940 raft_consensus.cc:504] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:36.338163 7979 raft_consensus.cc:504] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:36.338510 7979 leader_election.cc:288] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 2 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
W20240502 20:46:36.339406 31691 leader_election.cc:334] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.339453 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5bdf4f57d2c4c828b94c7cc2e3c2847" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:36.339536 31755 raft_consensus.cc:3037] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Advancing to term 2
W20240502 20:46:36.340107 29118 leader_election.cc:334] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.340185 31755 raft_consensus.cc:2454] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P e7ab6b9053664ee5aa71c7978e26fe6a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 2.
I20240502 20:46:36.340490 29117 leader_election.cc:302] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters: 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:36.340710 7940 leader_election.cc:288] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 2 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:36.340960 7979 raft_consensus.cc:2781] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:36.340993 31494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d5cca35ba4b47748235d1463ec161d6" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:36.341048 7979 raft_consensus.cc:686] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d [term 2 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:36.341092 31494 raft_consensus.cc:3037] T 8d5cca35ba4b47748235d1463ec161d6 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:36.341173 7979 consensus_queue.cc:237] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:36.342018 31494 raft_consensus.cc:2454] T 8d5cca35ba4b47748235d1463ec161d6 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 2.
I20240502 20:46:36.342330 31692 leader_election.cc:302] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e7ab6b9053664ee5aa71c7978e26fe6a; no voters: 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:36.342578 7940 raft_consensus.cc:2781] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:36.342657 7940 raft_consensus.cc:686] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a [term 2 LEADER]: Becoming Leader. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Running, Role: LEADER
I20240502 20:46:36.342762 7940 consensus_queue.cc:237] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:36.344028 29040 catalog_manager.cc:5484] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 1 to 2, leader changed from 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 2 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:36.344733 29036 catalog_manager.cc:5484] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: term changed from 1 to 2, leader changed from 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) to e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71). New cstate: current_term: 2 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:36.346732 29175 raft_consensus.cc:1261] T fe41756b5dc74f7f9105e002b8bc9c84 P 7796ea3fba294d17a34386873969431d [term 2 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 20:46:36.347111 7910 consensus_queue.cc:1035] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 20:46:36.350808 29249 consensus_peers.cc:485] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:36.358487 7979 raft_consensus.cc:482] T 22571ccf8d3c4db98af98d75da3cb954 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:36.358565 7979 raft_consensus.cc:504] T 22571ccf8d3c4db98af98d75da3cb954 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:36.358937 7979 leader_election.cc:288] T 22571ccf8d3c4db98af98d75da3cb954 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:36.359232 31887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22571ccf8d3c4db98af98d75da3cb954" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:36.359270 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22571ccf8d3c4db98af98d75da3cb954" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:36.359359 31887 raft_consensus.cc:2454] T 22571ccf8d3c4db98af98d75da3cb954 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:36.359380 29444 raft_consensus.cc:2454] T 22571ccf8d3c4db98af98d75da3cb954 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:36.359686 29119 leader_election.cc:302] T 22571ccf8d3c4db98af98d75da3cb954 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:36.359839 7979 raft_consensus.cc:2781] T 22571ccf8d3c4db98af98d75da3cb954 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:36.359885 7979 raft_consensus.cc:482] T 22571ccf8d3c4db98af98d75da3cb954 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:36.359920 7979 raft_consensus.cc:3037] T 22571ccf8d3c4db98af98d75da3cb954 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:36.361042 7979 raft_consensus.cc:504] T 22571ccf8d3c4db98af98d75da3cb954 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:36.361380 7979 leader_election.cc:288] T 22571ccf8d3c4db98af98d75da3cb954 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095)
I20240502 20:46:36.362183 31887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22571ccf8d3c4db98af98d75da3cb954" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:36.362278 31887 raft_consensus.cc:3037] T 22571ccf8d3c4db98af98d75da3cb954 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:36.362882 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22571ccf8d3c4db98af98d75da3cb954" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:36.362998 29444 raft_consensus.cc:3037] T 22571ccf8d3c4db98af98d75da3cb954 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:36.363206 31887 raft_consensus.cc:2454] T 22571ccf8d3c4db98af98d75da3cb954 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:36.363932 29118 leader_election.cc:302] T 22571ccf8d3c4db98af98d75da3cb954 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:36.363999 29444 raft_consensus.cc:2454] T 22571ccf8d3c4db98af98d75da3cb954 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:36.364322 7239 raft_consensus.cc:2781] T 22571ccf8d3c4db98af98d75da3cb954 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:36.364393 7239 raft_consensus.cc:686] T 22571ccf8d3c4db98af98d75da3cb954 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:36.364495 7239 consensus_queue.cc:237] T 22571ccf8d3c4db98af98d75da3cb954 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:36.367609 29036 catalog_manager.cc:5484] T 22571ccf8d3c4db98af98d75da3cb954 P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
W20240502 20:46:36.394690 31691 consensus_peers.cc:462] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.395574 29444 raft_consensus.cc:1261] T 5993321dc8f54f14a89fa2ba4a44ccc7 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 2 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 20:46:36.395951 7829 consensus_queue.cc:1035] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:36.452600 29036 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:36.456947 31887 raft_consensus.cc:1261] T 1e16a4014cf24e84abc7a7d42460dea3 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 20:46:36.457159 29118 consensus_peers.cc:485] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:36.457397 7239 consensus_queue.cc:1035] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:36.460435 8133 tablet.cc:1722] T 1e16a4014cf24e84abc7a7d42460dea3 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:36.460793 8133 tablet.cc:1620] T 1e16a4014cf24e84abc7a7d42460dea3 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:36.461026 8132 tablet.cc:1722] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:36.461323 8132 tablet.cc:1620] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d: MemRowSet was empty: no flush needed.
I20240502 20:46:36.461766 29036 catalog_manager.cc:5576] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207) does not have the latest schema for tablet 1e16a4014cf24e84abc7a7d42460dea3 (table rebalancer_dynamic_table [id=089f58cd5abe45a7bcc326ddd6802ab3]). Expected version 1 got 0
I20240502 20:46:36.467972 29036 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=089f58cd5abe45a7bcc326ddd6802ab3] alter complete (version 1)
I20240502 20:46:36.471493 31755 raft_consensus.cc:1261] T 90bab3d11d424b3abcdc74d5da51cbee P e7ab6b9053664ee5aa71c7978e26fe6a [term 2 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 20:46:36.471870 6234 consensus_queue.cc:1035] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 20:46:36.474946 29380 consensus_peers.cc:485] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:36.475849 31887 raft_consensus.cc:1261] T 0f5e277eec15411c8c1fd76189893ee9 P e6b60803d0764d228c863e2da65e3691 [term 2 FOLLOWER]: Refusing update from remote peer e7ab6b9053664ee5aa71c7978e26fe6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 20:46:36.476225 7200 consensus_queue.cc:1035] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 20:46:36.479383 31691 consensus_peers.cc:485] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:36.495493 6234 raft_consensus.cc:482] T 40305eb373a542f4a6aba1a55947442b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:36.495635 6234 raft_consensus.cc:504] T 40305eb373a542f4a6aba1a55947442b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:36.495995 6234 leader_election.cc:288] T 40305eb373a542f4a6aba1a55947442b P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:36.496300 31887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40305eb373a542f4a6aba1a55947442b" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:36.496418 31887 raft_consensus.cc:2454] T 40305eb373a542f4a6aba1a55947442b P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:36.496699 29380 leader_election.cc:302] T 40305eb373a542f4a6aba1a55947442b P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dbcdc6ac635b44b185c3eba63f66e9d6, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:36.496863 6234 raft_consensus.cc:2781] T 40305eb373a542f4a6aba1a55947442b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:36.496932 6234 raft_consensus.cc:482] T 40305eb373a542f4a6aba1a55947442b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:36.496979 6234 raft_consensus.cc:3037] T 40305eb373a542f4a6aba1a55947442b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
W20240502 20:46:36.497187 29380 leader_election.cc:334] T 40305eb373a542f4a6aba1a55947442b P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.497844 6234 raft_consensus.cc:504] T 40305eb373a542f4a6aba1a55947442b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:36.498150 6234 leader_election.cc:288] T 40305eb373a542f4a6aba1a55947442b P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:36.498415 31887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40305eb373a542f4a6aba1a55947442b" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:36.498538 31887 raft_consensus.cc:3037] T 40305eb373a542f4a6aba1a55947442b P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
W20240502 20:46:36.499022 29380 leader_election.cc:334] T 40305eb373a542f4a6aba1a55947442b P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:36.499356 31887 raft_consensus.cc:2454] T 40305eb373a542f4a6aba1a55947442b P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:36.499836 29380 leader_election.cc:302] T 40305eb373a542f4a6aba1a55947442b P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: dbcdc6ac635b44b185c3eba63f66e9d6, e6b60803d0764d228c863e2da65e3691; no voters: 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:36.500083 6234 raft_consensus.cc:2781] T 40305eb373a542f4a6aba1a55947442b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:36.500180 6234 raft_consensus.cc:686] T 40305eb373a542f4a6aba1a55947442b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:36.500308 6234 consensus_queue.cc:237] T 40305eb373a542f4a6aba1a55947442b P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:36.503240 29036 catalog_manager.cc:5484] T 40305eb373a542f4a6aba1a55947442b P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:36.517671 29295 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:36.522547 29163 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 20:46:36.527424 31867 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:36.530611 31475 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240502 20:46:36.530612 29557 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 20:46:36.549891 29424 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 20:46:36.550139 31755 raft_consensus.cc:1261] T b15554a988034cc7ab18ac2330f1a356 P e7ab6b9053664ee5aa71c7978e26fe6a [term 2 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 20:46:36.550647 7256 consensus_queue.cc:1035] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:36.558634 31735 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 20:46:36.571322 29036 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
W20240502 20:46:36.577363 29511 consensus_peers.cc:485] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:36.578069 8132 tablet.cc:1722] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:36.578383 8132 tablet.cc:1620] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d: MemRowSet was empty: no flush needed.
I20240502 20:46:36.578898 8133 tablet.cc:1722] T 1e16a4014cf24e84abc7a7d42460dea3 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:36.579170 8133 tablet.cc:1620] T 1e16a4014cf24e84abc7a7d42460dea3 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:36.583629 29039 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=089f58cd5abe45a7bcc326ddd6802ab3] alter complete (version 2)
W20240502 20:46:36.586515 31822 consensus_peers.cc:485] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:36.604854 29511 consensus_peers.cc:485] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:36.607331 31495 consensus_queue.cc:237] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:36.611037 29175 raft_consensus.cc:1261] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:36.611101 31887 raft_consensus.cc:1261] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:36.611444 7829 consensus_queue.cc:1035] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:36.611631 6616 consensus_queue.cc:1035] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:36.613353 7829 raft_consensus.cc:2932] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:36.613910 31887 raft_consensus.cc:2932] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:36.616493 29039 catalog_manager.cc:5484] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: config changed from index 2 to 3, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) evicted. New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:36.616714 29175 raft_consensus.cc:2932] T 2227f1ae58a044ea955cffa8d8d8a3f2 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:36.619395 31735 tablet_service.cc:1514] Processing DeleteTablet for tablet 2227f1ae58a044ea955cffa8d8d8a3f2 with delete_type TABLET_DATA_TOMBSTONED (TS e7ab6b9053664ee5aa71c7978e26fe6a not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:33864
I20240502 20:46:36.620469 8169 tablet_replica.cc:330] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e7ab6b9053664ee5aa71c7978e26fe6a: stopping tablet replica
I20240502 20:46:36.620682 8169 raft_consensus.cc:2229] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:36.620844 8169 raft_consensus.cc:2258] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:36.621310 8169 ts_tablet_manager.cc:1875] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:36.623754 8169 ts_tablet_manager.cc:1888] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 20:46:36.623819 8169 log.cc:1196] T 2227f1ae58a044ea955cffa8d8d8a3f2 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/wals/2227f1ae58a044ea955cffa8d8d8a3f2
I20240502 20:46:36.624310 29026 catalog_manager.cc:4830] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): tablet 2227f1ae58a044ea955cffa8d8d8a3f2 (table rebalancer_extra_table_26 [id=288d696f58ef4fb08ffcc029da0478db]) successfully deleted
I20240502 20:46:36.686864 29033 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:36.693476 29574 raft_consensus.cc:1261] T 03db15f1ebfb4e4ea271d6bb5c76110d P db17f26a73f3447b92aa9a947e3a7057 [term 2 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 20:46:36.693841 6850 consensus_queue.cc:1035] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:36.698343 8132 tablet.cc:1722] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:36.704649 8133 tablet.cc:1722] T 1e16a4014cf24e84abc7a7d42460dea3 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:36.704998 29033 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=089f58cd5abe45a7bcc326ddd6802ab3] alter complete (version 3)
I20240502 20:46:36.709801 31755 raft_consensus.cc:1261] T 58d4026c208f45d8870a2eaa1c97efab P e7ab6b9053664ee5aa71c7978e26fe6a [term 2 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 20:46:36.710516 5346 consensus_queue.cc:1035] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:36.749085 7256 consensus_queue.cc:1035] T ab91f7c07fd14fbb8cd8a70bf8f616ab P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:36.749362 29295 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:36.752312 8211 consensus_queue.cc:1035] T 22571ccf8d3c4db98af98d75da3cb954 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:36.754441 29163 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 20:46:36.757265 8211 consensus_queue.cc:1035] T 22571ccf8d3c4db98af98d75da3cb954 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:36.762610 29557 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 20:46:36.762966 29424 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 20:46:36.765487 31735 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 20:46:36.766384 31755 raft_consensus.cc:1261] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P e7ab6b9053664ee5aa71c7978e26fe6a [term 2 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 20:46:36.766851 8211 consensus_queue.cc:1035] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:36.768779 31867 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:36.773919 31475 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
W20240502 20:46:36.784164 29118 consensus_peers.cc:485] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:36.787587 29511 consensus_peers.cc:485] T ab91f7c07fd14fbb8cd8a70bf8f616ab P db17f26a73f3447b92aa9a947e3a7057 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:36.788534 31887 raft_consensus.cc:1261] T 6e80723ee7624d698e9e0a0bd494e1f0 P e6b60803d0764d228c863e2da65e3691 [term 2 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 20:46:36.788908 7829 consensus_queue.cc:1035] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 20:46:36.793367 31429 consensus_peers.cc:485] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:36.797272 29182 raft_consensus.cc:1261] T 358e4c761223404fbf0f467f38ce9b25 P 7796ea3fba294d17a34386873969431d [term 2 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240502 20:46:36.797714 7829 consensus_queue.cc:1035] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:36.800585 31757 raft_consensus.cc:1261] T 358e4c761223404fbf0f467f38ce9b25 P e7ab6b9053664ee5aa71c7978e26fe6a [term 2 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240502 20:46:36.801002 7829 consensus_queue.cc:1035] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20240502 20:46:36.801853 31691 consensus_peers.cc:462] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
W20240502 20:46:36.809974 31429 consensus_peers.cc:485] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:36.814985 29040 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:36.815116 29040 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:36.817135 29040 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 1e16a4014cf24e84abc7a7d42460dea3
I20240502 20:46:36.817703 31867 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e16a4014cf24e84abc7a7d42460dea3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:36 UTC) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:36.818985 29163 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e16a4014cf24e84abc7a7d42460dea3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:36 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:36.820564 8232 tablet_replica.cc:330] T 1e16a4014cf24e84abc7a7d42460dea3 P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:36.820741 8232 raft_consensus.cc:2229] T 1e16a4014cf24e84abc7a7d42460dea3 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:36.820906 8232 raft_consensus.cc:2258] T 1e16a4014cf24e84abc7a7d42460dea3 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:36.821739 8232 ts_tablet_manager.cc:1875] T 1e16a4014cf24e84abc7a7d42460dea3 P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:36.822618 8233 tablet_replica.cc:330] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:36.822847 8233 raft_consensus.cc:2229] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:36.823235 8233 raft_consensus.cc:2258] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:36.823855 8233 ts_tablet_manager.cc:1875] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:36.825182 8232 ts_tablet_manager.cc:1888] T 1e16a4014cf24e84abc7a7d42460dea3 P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:36.825304 8232 log.cc:1196] T 1e16a4014cf24e84abc7a7d42460dea3 P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/1e16a4014cf24e84abc7a7d42460dea3
I20240502 20:46:36.825650 8232 ts_tablet_manager.cc:1909] T 1e16a4014cf24e84abc7a7d42460dea3 P e6b60803d0764d228c863e2da65e3691: Deleting consensus metadata
I20240502 20:46:36.826565 8233 ts_tablet_manager.cc:1888] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:36.826673 8233 log.cc:1196] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/1e16a4014cf24e84abc7a7d42460dea3
I20240502 20:46:36.826879 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet 1e16a4014cf24e84abc7a7d42460dea3 (table rebalancer_dynamic_table_new_name [id=089f58cd5abe45a7bcc326ddd6802ab3]) successfully deleted
I20240502 20:46:36.827047 8233 ts_tablet_manager.cc:1909] T 1e16a4014cf24e84abc7a7d42460dea3 P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:36.827683 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 1e16a4014cf24e84abc7a7d42460dea3 (table rebalancer_dynamic_table_new_name [id=089f58cd5abe45a7bcc326ddd6802ab3]) successfully deleted
I20240502 20:46:36.859704 29315 consensus_queue.cc:237] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:36.860814 31757 raft_consensus.cc:1261] T 43fc3dcc2dd54a11becb8f16ec500df1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:36.861132 29444 raft_consensus.cc:1261] T 43fc3dcc2dd54a11becb8f16ec500df1 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer 09258ac095664fa99caf0707262cde13: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:36.861802 7181 consensus_queue.cc:1035] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:36.862269 7181 consensus_queue.cc:1035] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:36.864089 7910 raft_consensus.cc:2932] T 43fc3dcc2dd54a11becb8f16ec500df1 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:36.864472 29444 raft_consensus.cc:2932] T 43fc3dcc2dd54a11becb8f16ec500df1 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:36.867615 29040 catalog_manager.cc:5484] T 43fc3dcc2dd54a11becb8f16ec500df1 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: config changed from index 2 to 3, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) evicted. New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:36.870087 31867 tablet_service.cc:1514] Processing DeleteTablet for tablet 43fc3dcc2dd54a11becb8f16ec500df1 with delete_type TABLET_DATA_TOMBSTONED (TS e6b60803d0764d228c863e2da65e3691 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:36.870481 8232 tablet_replica.cc:330] T 43fc3dcc2dd54a11becb8f16ec500df1 P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:36.870641 8232 raft_consensus.cc:2229] T 43fc3dcc2dd54a11becb8f16ec500df1 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:36.870761 8232 raft_consensus.cc:2258] T 43fc3dcc2dd54a11becb8f16ec500df1 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:36.871168 8232 ts_tablet_manager.cc:1875] T 43fc3dcc2dd54a11becb8f16ec500df1 P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:36.873126 8232 ts_tablet_manager.cc:1888] T 43fc3dcc2dd54a11becb8f16ec500df1 P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 20:46:36.875349 8232 log.cc:1196] T 43fc3dcc2dd54a11becb8f16ec500df1 P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/43fc3dcc2dd54a11becb8f16ec500df1
I20240502 20:46:36.875721 31757 raft_consensus.cc:2932] T 43fc3dcc2dd54a11becb8f16ec500df1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e6b60803d0764d228c863e2da65e3691 (127.22.45.72) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:36.875842 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet 43fc3dcc2dd54a11becb8f16ec500df1 (table rebalancer_extra_table_32 [id=a863244f6b74429ba652c196f42b4ee0]) successfully deleted
I20240502 20:46:36.892206 31493 raft_consensus.cc:1261] T 8d5cca35ba4b47748235d1463ec161d6 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 2 FOLLOWER]: Refusing update from remote peer e7ab6b9053664ee5aa71c7978e26fe6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 20:46:36.892590 7940 consensus_queue.cc:1035] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 20:46:36.896868 31429 consensus_peers.cc:485] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:36.896903 29575 raft_consensus.cc:1261] T 08feee781db04883b6dcc030bdaa8afe P db17f26a73f3447b92aa9a947e3a7057 [term 2 FOLLOWER]: Refusing update from remote peer 2a9c1af8cf814030bde56b67ca5c2fbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20240502 20:46:36.897272 31691 consensus_peers.cc:485] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:36.897287 8116 consensus_queue.cc:1035] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:36.920729 29040 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:36.927369 29163 tablet_service.cc:1467] Processing CreateTablet for tablet 945af26fb3584f4d9f6a8c10438a0eca (DEFAULT_TABLE table=rebalancer_dynamic_table [id=f4b6b521d94e400a9111ca8c01764950]), partition=
I20240502 20:46:36.927608 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 945af26fb3584f4d9f6a8c10438a0eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:36.928922 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 945af26fb3584f4d9f6a8c10438a0eca (DEFAULT_TABLE table=rebalancer_dynamic_table [id=f4b6b521d94e400a9111ca8c01764950]), partition=
I20240502 20:46:36.929147 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 945af26fb3584f4d9f6a8c10438a0eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:36.932039 8296 tablet_bootstrap.cc:492] T 945af26fb3584f4d9f6a8c10438a0eca P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:36.933357 8296 tablet_bootstrap.cc:654] T 945af26fb3584f4d9f6a8c10438a0eca P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:36.934244 8297 tablet_bootstrap.cc:492] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:36.935262 8296 tablet_bootstrap.cc:492] T 945af26fb3584f4d9f6a8c10438a0eca P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:36.935276 8297 tablet_bootstrap.cc:654] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:36.935349 8296 ts_tablet_manager.cc:1367] T 945af26fb3584f4d9f6a8c10438a0eca P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 20:46:36.935755 8296 raft_consensus.cc:348] T 945af26fb3584f4d9f6a8c10438a0eca P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:36.935983 8296 raft_consensus.cc:374] T 945af26fb3584f4d9f6a8c10438a0eca P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:36.936074 8296 raft_consensus.cc:729] T 945af26fb3584f4d9f6a8c10438a0eca P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:36.936210 8296 consensus_queue.cc:260] T 945af26fb3584f4d9f6a8c10438a0eca P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:36.936389 8297 tablet_bootstrap.cc:492] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:36.936480 8296 ts_tablet_manager.cc:1398] T 945af26fb3584f4d9f6a8c10438a0eca P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:36.936491 8297 ts_tablet_manager.cc:1367] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:36.936903 8297 raft_consensus.cc:348] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:36.937072 8297 raft_consensus.cc:374] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:36.937541 8297 raft_consensus.cc:729] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:36.937744 8297 consensus_queue.cc:260] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:36.938053 8297 ts_tablet_manager.cc:1398] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
W20240502 20:46:36.963567 29027 catalog_manager.cc:4573] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): CreateTablet RPC failed for tablet 945af26fb3584f4d9f6a8c10438a0eca: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:37.003808 6234 consensus_queue.cc:1035] T 40305eb373a542f4a6aba1a55947442b P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 20:46:37.012178 29380 consensus_peers.cc:485] T 40305eb373a542f4a6aba1a55947442b P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:37.021731 29163 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 20:46:37.024755 31475 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240502 20:46:37.027105 8129 raft_consensus.cc:482] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:37.027200 8129 raft_consensus.cc:504] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:37.027542 8129 leader_election.cc:288] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:37.030404 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "945af26fb3584f4d9f6a8c10438a0eca" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:37.030553 29182 raft_consensus.cc:2454] T 945af26fb3584f4d9f6a8c10438a0eca P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
W20240502 20:46:37.030769 31822 leader_election.cc:334] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:37.030995 31824 leader_election.cc:302] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e6b60803d0764d228c863e2da65e3691; no voters: 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:37.031371 7076 raft_consensus.cc:2781] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:37.031438 7076 raft_consensus.cc:482] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:37.031476 7076 raft_consensus.cc:3037] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:37.032552 7076 raft_consensus.cc:504] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:37.032884 7076 leader_election.cc:288] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:37.033083 29557 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 20:46:37.033917 29295 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:37.038709 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "945af26fb3584f4d9f6a8c10438a0eca" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:37.038820 29182 raft_consensus.cc:3037] T 945af26fb3584f4d9f6a8c10438a0eca P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:37.039161 29424 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 20:46:37.039878 29182 raft_consensus.cc:2454] T 945af26fb3584f4d9f6a8c10438a0eca P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:37.040158 31824 leader_election.cc:302] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:37.040481 6850 raft_consensus.cc:2781] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:37.040575 6850 raft_consensus.cc:686] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Becoming Leader. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Running, Role: LEADER
I20240502 20:46:37.040697 6850 consensus_queue.cc:237] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:37.043036 29041 catalog_manager.cc:5484] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6b60803d0764d228c863e2da65e3691 (127.22.45.72). New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:37.043478 29039 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_62"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:37.050096 29557 tablet_service.cc:1467] Processing CreateTablet for tablet 0681d19967c74f3da72497366fd9e3fb (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=8c206a00bffc41859730ddd69832c891]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:37.050370 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0681d19967c74f3da72497366fd9e3fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:37.050750 31736 tablet_service.cc:1467] Processing CreateTablet for tablet 5156cc195fcd4b28b9c7bf1aa8a1409e (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=8c206a00bffc41859730ddd69832c891]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:37.050776 31735 tablet_service.cc:1467] Processing CreateTablet for tablet 0681d19967c74f3da72497366fd9e3fb (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=8c206a00bffc41859730ddd69832c891]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:37.051035 31736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5156cc195fcd4b28b9c7bf1aa8a1409e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:37.051182 31735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0681d19967c74f3da72497366fd9e3fb. 1 dirs total, 0 dirs full, 0 dirs failed
W20240502 20:46:37.051779 31822 leader_election.cc:334] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:37.052753 8313 tablet_bootstrap.cc:492] T 0681d19967c74f3da72497366fd9e3fb P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:37.052914 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 0681d19967c74f3da72497366fd9e3fb (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=8c206a00bffc41859730ddd69832c891]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:37.053139 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0681d19967c74f3da72497366fd9e3fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:37.053651 8313 tablet_bootstrap.cc:654] T 0681d19967c74f3da72497366fd9e3fb P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:37.055220 8297 tablet_bootstrap.cc:492] T 0681d19967c74f3da72497366fd9e3fb P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:37.055308 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 87670bbef3884cc79a7c7e0533661c12 (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=8c206a00bffc41859730ddd69832c891]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:37.055477 8314 tablet_bootstrap.cc:492] T 0681d19967c74f3da72497366fd9e3fb P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:37.055539 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87670bbef3884cc79a7c7e0533661c12. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:37.056082 8297 tablet_bootstrap.cc:654] T 0681d19967c74f3da72497366fd9e3fb P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:37.056679 8314 tablet_bootstrap.cc:654] T 0681d19967c74f3da72497366fd9e3fb P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:37.057467 29556 tablet_service.cc:1467] Processing CreateTablet for tablet 87670bbef3884cc79a7c7e0533661c12 (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=8c206a00bffc41859730ddd69832c891]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:37.057713 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87670bbef3884cc79a7c7e0533661c12. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:37.058120 8297 tablet_bootstrap.cc:492] T 0681d19967c74f3da72497366fd9e3fb P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:37.058207 8297 ts_tablet_manager.cc:1367] T 0681d19967c74f3da72497366fd9e3fb P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:37.058689 8297 raft_consensus.cc:348] T 0681d19967c74f3da72497366fd9e3fb P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:37.058782 8297 raft_consensus.cc:374] T 0681d19967c74f3da72497366fd9e3fb P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:37.058815 8297 raft_consensus.cc:729] T 0681d19967c74f3da72497366fd9e3fb P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:37.058919 8297 consensus_queue.cc:260] T 0681d19967c74f3da72497366fd9e3fb P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:37.059171 8297 ts_tablet_manager.cc:1398] T 0681d19967c74f3da72497366fd9e3fb P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:37.057484 29557 tablet_service.cc:1467] Processing CreateTablet for tablet 5156cc195fcd4b28b9c7bf1aa8a1409e (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=8c206a00bffc41859730ddd69832c891]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:37.064025 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5156cc195fcd4b28b9c7bf1aa8a1409e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:37.065389 8313 tablet_bootstrap.cc:492] T 0681d19967c74f3da72497366fd9e3fb P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:37.065941 8313 ts_tablet_manager.cc:1367] T 0681d19967c74f3da72497366fd9e3fb P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 20:46:37.066815 8318 tablet_bootstrap.cc:492] T 87670bbef3884cc79a7c7e0533661c12 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:37.067780 31736 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 20:46:37.070845 31867 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:37.072960 8314 tablet_bootstrap.cc:492] T 0681d19967c74f3da72497366fd9e3fb P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:37.073089 8314 ts_tablet_manager.cc:1367] T 0681d19967c74f3da72497366fd9e3fb P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240502 20:46:37.073478 8314 raft_consensus.cc:348] T 0681d19967c74f3da72497366fd9e3fb P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:37.073633 8314 raft_consensus.cc:374] T 0681d19967c74f3da72497366fd9e3fb P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:37.073696 8314 raft_consensus.cc:729] T 0681d19967c74f3da72497366fd9e3fb P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:37.073854 8314 consensus_queue.cc:260] T 0681d19967c74f3da72497366fd9e3fb P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:37.075088 8314 ts_tablet_manager.cc:1398] T 0681d19967c74f3da72497366fd9e3fb P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:37.075274 8314 tablet_bootstrap.cc:492] T 5156cc195fcd4b28b9c7bf1aa8a1409e P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:37.076164 8314 tablet_bootstrap.cc:654] T 5156cc195fcd4b28b9c7bf1aa8a1409e P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:37.078178 8314 tablet_bootstrap.cc:492] T 5156cc195fcd4b28b9c7bf1aa8a1409e P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:37.078249 8314 ts_tablet_manager.cc:1367] T 5156cc195fcd4b28b9c7bf1aa8a1409e P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:37.078754 8314 raft_consensus.cc:348] T 5156cc195fcd4b28b9c7bf1aa8a1409e P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:37.078845 8314 raft_consensus.cc:374] T 5156cc195fcd4b28b9c7bf1aa8a1409e P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:37.078872 8314 raft_consensus.cc:729] T 5156cc195fcd4b28b9c7bf1aa8a1409e P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:37.078967 8314 consensus_queue.cc:260] T 5156cc195fcd4b28b9c7bf1aa8a1409e P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:37.079159 8314 ts_tablet_manager.cc:1398] T 5156cc195fcd4b28b9c7bf1aa8a1409e P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:37.080839 8318 tablet_bootstrap.cc:654] T 87670bbef3884cc79a7c7e0533661c12 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:37.082155 8318 tablet_bootstrap.cc:492] T 87670bbef3884cc79a7c7e0533661c12 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:37.082290 8318 ts_tablet_manager.cc:1367] T 87670bbef3884cc79a7c7e0533661c12 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240502 20:46:37.082810 8318 raft_consensus.cc:348] T 87670bbef3884cc79a7c7e0533661c12 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:37.082973 8318 raft_consensus.cc:374] T 87670bbef3884cc79a7c7e0533661c12 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:37.083063 8318 raft_consensus.cc:729] T 87670bbef3884cc79a7c7e0533661c12 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:37.083251 8318 consensus_queue.cc:260] T 87670bbef3884cc79a7c7e0533661c12 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:37.083609 8318 ts_tablet_manager.cc:1398] T 87670bbef3884cc79a7c7e0533661c12 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:37.102972 8313 raft_consensus.cc:348] T 0681d19967c74f3da72497366fd9e3fb P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:37.103092 8313 raft_consensus.cc:374] T 0681d19967c74f3da72497366fd9e3fb P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:37.103145 8313 raft_consensus.cc:729] T 0681d19967c74f3da72497366fd9e3fb P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:37.103271 8313 consensus_queue.cc:260] T 0681d19967c74f3da72497366fd9e3fb P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:37.103616 8313 ts_tablet_manager.cc:1398] T 0681d19967c74f3da72497366fd9e3fb P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:37.103920 8313 tablet_bootstrap.cc:492] T 87670bbef3884cc79a7c7e0533661c12 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:37.105021 8313 tablet_bootstrap.cc:654] T 87670bbef3884cc79a7c7e0533661c12 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:37.106067 8313 tablet_bootstrap.cc:492] T 87670bbef3884cc79a7c7e0533661c12 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:37.106143 8313 ts_tablet_manager.cc:1367] T 87670bbef3884cc79a7c7e0533661c12 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:37.106582 8313 raft_consensus.cc:348] T 87670bbef3884cc79a7c7e0533661c12 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:37.106680 8313 raft_consensus.cc:374] T 87670bbef3884cc79a7c7e0533661c12 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:37.106706 8313 raft_consensus.cc:729] T 87670bbef3884cc79a7c7e0533661c12 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:37.106806 8313 consensus_queue.cc:260] T 87670bbef3884cc79a7c7e0533661c12 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:37.107050 8313 ts_tablet_manager.cc:1398] T 87670bbef3884cc79a7c7e0533661c12 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:37.107192 8313 tablet_bootstrap.cc:492] T 5156cc195fcd4b28b9c7bf1aa8a1409e P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:37.108028 8313 tablet_bootstrap.cc:654] T 5156cc195fcd4b28b9c7bf1aa8a1409e P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:37.109124 8313 tablet_bootstrap.cc:492] T 5156cc195fcd4b28b9c7bf1aa8a1409e P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:37.109215 8313 ts_tablet_manager.cc:1367] T 5156cc195fcd4b28b9c7bf1aa8a1409e P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:37.109632 8313 raft_consensus.cc:348] T 5156cc195fcd4b28b9c7bf1aa8a1409e P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:37.109730 8313 raft_consensus.cc:374] T 5156cc195fcd4b28b9c7bf1aa8a1409e P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:37.109766 8313 raft_consensus.cc:729] T 5156cc195fcd4b28b9c7bf1aa8a1409e P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:37.109867 8313 consensus_queue.cc:260] T 5156cc195fcd4b28b9c7bf1aa8a1409e P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:37.110119 8313 ts_tablet_manager.cc:1398] T 5156cc195fcd4b28b9c7bf1aa8a1409e P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:37.170295 29036 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:37.175369 29182 raft_consensus.cc:1261] T 945af26fb3584f4d9f6a8c10438a0eca P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:37.175953 7076 consensus_queue.cc:1035] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 20:46:37.177548 31822 consensus_peers.cc:485] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:37.179677 8133 tablet.cc:1722] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:37.179991 8133 tablet.cc:1620] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:37.181442 29041 catalog_manager.cc:5576] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267) does not have the latest schema for tablet 945af26fb3584f4d9f6a8c10438a0eca (table rebalancer_dynamic_table [id=f4b6b521d94e400a9111ca8c01764950]). Expected version 1 got 0
I20240502 20:46:37.182991 8132 tablet.cc:1722] T 945af26fb3584f4d9f6a8c10438a0eca P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:37.183291 8132 tablet.cc:1620] T 945af26fb3584f4d9f6a8c10438a0eca P 7796ea3fba294d17a34386873969431d: MemRowSet was empty: no flush needed.
I20240502 20:46:37.183581 7256 raft_consensus.cc:482] T 87670bbef3884cc79a7c7e0533661c12 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:37.183672 7256 raft_consensus.cc:504] T 87670bbef3884cc79a7c7e0533661c12 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:37.184024 7256 leader_election.cc:288] T 87670bbef3884cc79a7c7e0533661c12 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:37.184403 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87670bbef3884cc79a7c7e0533661c12" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:37.184554 29315 raft_consensus.cc:2454] T 87670bbef3884cc79a7c7e0533661c12 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:37.184839 29512 leader_election.cc:302] T 87670bbef3884cc79a7c7e0533661c12 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057; no voters:
W20240502 20:46:37.185534 29380 consensus_peers.cc:485] T 653cbec396434b4dac0a1928b21d0ebd P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 20:46:37.186573 29511 leader_election.cc:334] T 87670bbef3884cc79a7c7e0533661c12 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
W20240502 20:46:37.186717 29511 consensus_peers.cc:485] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Error code: TABLET_NOT_FOUND (6). Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:37.187126 5346 raft_consensus.cc:2781] T 87670bbef3884cc79a7c7e0533661c12 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:37.187193 5346 raft_consensus.cc:482] T 87670bbef3884cc79a7c7e0533661c12 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:37.187232 5346 raft_consensus.cc:3037] T 87670bbef3884cc79a7c7e0533661c12 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:37.188091 5346 raft_consensus.cc:504] T 87670bbef3884cc79a7c7e0533661c12 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:37.188400 5346 leader_election.cc:288] T 87670bbef3884cc79a7c7e0533661c12 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Requested vote from peers 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:37.188666 29036 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=f4b6b521d94e400a9111ca8c01764950] alter complete (version 1)
I20240502 20:46:37.188896 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87670bbef3884cc79a7c7e0533661c12" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:37.188994 29315 raft_consensus.cc:3037] T 87670bbef3884cc79a7c7e0533661c12 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:37.189903 29315 raft_consensus.cc:2454] T 87670bbef3884cc79a7c7e0533661c12 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
W20240502 20:46:37.192755 29511 leader_election.cc:334] T 87670bbef3884cc79a7c7e0533661c12 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:37.192878 29512 leader_election.cc:302] T 87670bbef3884cc79a7c7e0533661c12 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09258ac095664fa99caf0707262cde13, db17f26a73f3447b92aa9a947e3a7057; no voters: 315915efcf2b413ca7f9d874338e1b2d
I20240502 20:46:37.193248 5346 raft_consensus.cc:2781] T 87670bbef3884cc79a7c7e0533661c12 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:37.193318 5346 raft_consensus.cc:686] T 87670bbef3884cc79a7c7e0533661c12 P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Becoming Leader. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Running, Role: LEADER
I20240502 20:46:37.193425 5346 consensus_queue.cc:237] T 87670bbef3884cc79a7c7e0533661c12 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } }
I20240502 20:46:37.196306 29036 catalog_manager.cc:5484] T 87670bbef3884cc79a7c7e0533661c12 P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: term changed from 0 to 1, leader changed from <none> to db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } }
W20240502 20:46:37.210357 31691 consensus_peers.cc:485] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Error code: TABLET_NOT_FOUND (6). Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:37.221118 8264 raft_consensus.cc:482] T 5156cc195fcd4b28b9c7bf1aa8a1409e P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:37.221210 8264 raft_consensus.cc:504] T 5156cc195fcd4b28b9c7bf1aa8a1409e P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:37.221561 8264 leader_election.cc:288] T 5156cc195fcd4b28b9c7bf1aa8a1409e P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:37.222043 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5156cc195fcd4b28b9c7bf1aa8a1409e" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:37.222151 29575 raft_consensus.cc:2454] T 5156cc195fcd4b28b9c7bf1aa8a1409e P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:37.222724 31691 leader_election.cc:302] T 5156cc195fcd4b28b9c7bf1aa8a1409e P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
W20240502 20:46:37.222893 31691 leader_election.cc:334] T 5156cc195fcd4b28b9c7bf1aa8a1409e P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:37.222903 8264 raft_consensus.cc:2781] T 5156cc195fcd4b28b9c7bf1aa8a1409e P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:37.223009 8264 raft_consensus.cc:482] T 5156cc195fcd4b28b9c7bf1aa8a1409e P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:37.223062 8264 raft_consensus.cc:3037] T 5156cc195fcd4b28b9c7bf1aa8a1409e P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:37.224094 8264 raft_consensus.cc:504] T 5156cc195fcd4b28b9c7bf1aa8a1409e P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:37.224428 8264 leader_election.cc:288] T 5156cc195fcd4b28b9c7bf1aa8a1409e P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:37.226678 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5156cc195fcd4b28b9c7bf1aa8a1409e" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:37.226796 29575 raft_consensus.cc:3037] T 5156cc195fcd4b28b9c7bf1aa8a1409e P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:37.227819 29575 raft_consensus.cc:2454] T 5156cc195fcd4b28b9c7bf1aa8a1409e P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:37.229528 31691 leader_election.cc:302] T 5156cc195fcd4b28b9c7bf1aa8a1409e P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:37.230000 8264 raft_consensus.cc:2781] T 5156cc195fcd4b28b9c7bf1aa8a1409e P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:37.230094 8264 raft_consensus.cc:686] T 5156cc195fcd4b28b9c7bf1aa8a1409e P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Becoming Leader. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Running, Role: LEADER
I20240502 20:46:37.230228 8264 consensus_queue.cc:237] T 5156cc195fcd4b28b9c7bf1aa8a1409e P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
W20240502 20:46:37.230710 31691 leader_election.cc:334] T 5156cc195fcd4b28b9c7bf1aa8a1409e P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:37.233938 29038 catalog_manager.cc:5484] T 5156cc195fcd4b28b9c7bf1aa8a1409e P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: term changed from 0 to 1, leader changed from <none> to e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71). New cstate: current_term: 1 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
W20240502 20:46:37.236339 29380 consensus_peers.cc:485] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Error code: TABLET_NOT_FOUND (6). Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 20:46:37.239888 31691 consensus_peers.cc:485] T d6b1c1798aaf4f1fb3d6ef52503e08be P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 20:46:37.243217 31691 consensus_peers.cc:462] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
W20240502 20:46:37.253602 29511 consensus_peers.cc:485] T db5e418e67a44480b8d4c302cde3b96c P db17f26a73f3447b92aa9a947e3a7057 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 20:46:37.281718 29249 consensus_peers.cc:485] T f3c220fbd7df4cefb467ad88e99941b8 P 09258ac095664fa99caf0707262cde13 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:37.292881 6850 raft_consensus.cc:482] T 0681d19967c74f3da72497366fd9e3fb P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:37.292985 6850 raft_consensus.cc:504] T 0681d19967c74f3da72497366fd9e3fb P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:37.293326 6850 leader_election.cc:288] T 0681d19967c74f3da72497366fd9e3fb P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:37.293565 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0681d19967c74f3da72497366fd9e3fb" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:37.293690 29574 raft_consensus.cc:2454] T 0681d19967c74f3da72497366fd9e3fb P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:37.293705 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0681d19967c74f3da72497366fd9e3fb" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:37.293816 31755 raft_consensus.cc:2454] T 0681d19967c74f3da72497366fd9e3fb P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:37.294103 31821 leader_election.cc:302] T 0681d19967c74f3da72497366fd9e3fb P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e6b60803d0764d228c863e2da65e3691, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:37.294250 6850 raft_consensus.cc:2781] T 0681d19967c74f3da72497366fd9e3fb P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:37.294304 6850 raft_consensus.cc:482] T 0681d19967c74f3da72497366fd9e3fb P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:37.294338 6850 raft_consensus.cc:3037] T 0681d19967c74f3da72497366fd9e3fb P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:37.295385 29042 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:37.295614 6850 raft_consensus.cc:504] T 0681d19967c74f3da72497366fd9e3fb P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:37.295934 6850 leader_election.cc:288] T 0681d19967c74f3da72497366fd9e3fb P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:37.296212 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0681d19967c74f3da72497366fd9e3fb" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:37.296329 29574 raft_consensus.cc:3037] T 0681d19967c74f3da72497366fd9e3fb P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:37.297286 29574 raft_consensus.cc:2454] T 0681d19967c74f3da72497366fd9e3fb P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:37.297580 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0681d19967c74f3da72497366fd9e3fb" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:37.297652 31822 leader_election.cc:302] T 0681d19967c74f3da72497366fd9e3fb P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:37.297693 31755 raft_consensus.cc:3037] T 0681d19967c74f3da72497366fd9e3fb P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:37.297806 6850 raft_consensus.cc:2781] T 0681d19967c74f3da72497366fd9e3fb P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:37.297880 6850 raft_consensus.cc:686] T 0681d19967c74f3da72497366fd9e3fb P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Becoming Leader. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Running, Role: LEADER
I20240502 20:46:37.297994 6850 consensus_queue.cc:237] T 0681d19967c74f3da72497366fd9e3fb P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:37.298615 31755 raft_consensus.cc:2454] T 0681d19967c74f3da72497366fd9e3fb P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:37.300462 29042 catalog_manager.cc:5484] T 0681d19967c74f3da72497366fd9e3fb P e6b60803d0764d228c863e2da65e3691 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6b60803d0764d228c863e2da65e3691 (127.22.45.72). New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:37.301061 8133 tablet.cc:1722] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:37.301352 8133 tablet.cc:1620] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:37.303105 8132 tablet.cc:1722] T 945af26fb3584f4d9f6a8c10438a0eca P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:37.303382 8132 tablet.cc:1620] T 945af26fb3584f4d9f6a8c10438a0eca P 7796ea3fba294d17a34386873969431d: MemRowSet was empty: no flush needed.
I20240502 20:46:37.307914 29042 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=f4b6b521d94e400a9111ca8c01764950] alter complete (version 2)
I20240502 20:46:37.331400 29295 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 20:46:37.332975 31736 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 20:46:37.333186 31475 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240502 20:46:37.335354 29557 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 20:46:37.335960 29424 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 20:46:37.347131 31867 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 20:46:37.355147 29163 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 20:46:37.410696 29315 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "370bf1bcfb3343c488954ece2488b4c2"
dest_uuid: "09258ac095664fa99caf0707262cde13"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:37570
I20240502 20:46:37.410900 29315 raft_consensus.cc:595] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Received request to transfer leadership
I20240502 20:46:37.411828 29042 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
W20240502 20:46:37.414877 29380 consensus_peers.cc:485] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:37.417835 8133 tablet.cc:1722] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:37.418283 8132 tablet.cc:1722] T 945af26fb3584f4d9f6a8c10438a0eca P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:37.423847 29042 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=f4b6b521d94e400a9111ca8c01764950] alter complete (version 3)
W20240502 20:46:37.442188 29249 consensus_peers.cc:485] T 9992a7ea196645dc82d2430fcb405254 P 09258ac095664fa99caf0707262cde13 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 20:46:37.458029 31429 consensus_peers.cc:485] T d0538f41ea0e46edae87b8e4dfa995f0 P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 20:46:37.463868 29511 consensus_peers.cc:485] T 9da27cc9c2504b7899da253a88dd1f56 P db17f26a73f3447b92aa9a947e3a7057 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 20:46:37.464490 29249 consensus_peers.cc:485] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:37.465876 8406 raft_consensus.cc:979] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13: : Instructing follower 7796ea3fba294d17a34386873969431d to start an election
I20240502 20:46:37.465984 8406 raft_consensus.cc:1067] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Signalling peer 7796ea3fba294d17a34386873969431d to start an election
I20240502 20:46:37.466534 29182 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "370bf1bcfb3343c488954ece2488b4c2"
dest_uuid: "7796ea3fba294d17a34386873969431d"
from {username='slave'} at 127.22.45.66:56479
I20240502 20:46:37.466679 29182 raft_consensus.cc:482] T 370bf1bcfb3343c488954ece2488b4c2 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:37.466751 29182 raft_consensus.cc:3037] T 370bf1bcfb3343c488954ece2488b4c2 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:37.467932 29182 raft_consensus.cc:504] T 370bf1bcfb3343c488954ece2488b4c2 P 7796ea3fba294d17a34386873969431d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:37.468371 29182 leader_election.cc:288] T 370bf1bcfb3343c488954ece2488b4c2 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 2 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:37.468626 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "370bf1bcfb3343c488954ece2488b4c2" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:37.468744 29574 raft_consensus.cc:3037] T 370bf1bcfb3343c488954ece2488b4c2 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:37.468986 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "370bf1bcfb3343c488954ece2488b4c2" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:37.469097 31755 raft_consensus.cc:3037] T 370bf1bcfb3343c488954ece2488b4c2 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:37.469553 29574 raft_consensus.cc:2454] T 370bf1bcfb3343c488954ece2488b4c2 P db17f26a73f3447b92aa9a947e3a7057 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 2.
I20240502 20:46:37.470095 31755 raft_consensus.cc:2454] T 370bf1bcfb3343c488954ece2488b4c2 P e7ab6b9053664ee5aa71c7978e26fe6a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 2.
I20240502 20:46:37.470940 29117 leader_election.cc:302] T 370bf1bcfb3343c488954ece2488b4c2 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:37.471117 7979 raft_consensus.cc:2781] T 370bf1bcfb3343c488954ece2488b4c2 P 7796ea3fba294d17a34386873969431d [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:37.471194 7979 raft_consensus.cc:686] T 370bf1bcfb3343c488954ece2488b4c2 P 7796ea3fba294d17a34386873969431d [term 2 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:37.471309 7979 consensus_queue.cc:237] T 370bf1bcfb3343c488954ece2488b4c2 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:37.472769 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "370bf1bcfb3343c488954ece2488b4c2" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:37.472879 29315 raft_consensus.cc:3032] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:37.472930 29315 raft_consensus.cc:729] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:37.473076 29315 consensus_queue.cc:260] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:37.473243 29315 raft_consensus.cc:3037] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:37.474215 29315 raft_consensus.cc:2454] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 2.
I20240502 20:46:37.475440 29033 catalog_manager.cc:5484] T 370bf1bcfb3343c488954ece2488b4c2 P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 1 to 2, leader changed from 09258ac095664fa99caf0707262cde13 (127.22.45.66) to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 2 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
W20240502 20:46:37.490710 29118 consensus_peers.cc:485] T 3d89ea92aec54168997241dd78c6abb1 P 7796ea3fba294d17a34386873969431d -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 20:46:37.495908 31429 consensus_peers.cc:485] T 211fa6a23bda4c11bb3106080c3ace40 P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:37.527022 29038 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:37.527140 29038 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:37.529374 29038 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 945af26fb3584f4d9f6a8c10438a0eca
I20240502 20:46:37.530483 31867 tablet_service.cc:1514] Processing DeleteTablet for tablet 945af26fb3584f4d9f6a8c10438a0eca with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:37 UTC) from {username='slave'} at 127.0.0.1:40510
W20240502 20:46:37.533339 31429 consensus_peers.cc:485] T 2e4a0e5ef89d447b96b998f085e3495c P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:37.534206 29163 tablet_service.cc:1514] Processing DeleteTablet for tablet 945af26fb3584f4d9f6a8c10438a0eca with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:37 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:37.534444 8456 tablet_replica.cc:330] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:37.534796 8456 raft_consensus.cc:2229] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:37.535176 8456 raft_consensus.cc:2258] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:37.536617 8456 ts_tablet_manager.cc:1875] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:37.537704 8457 tablet_replica.cc:330] T 945af26fb3584f4d9f6a8c10438a0eca P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:37.537963 8457 raft_consensus.cc:2229] T 945af26fb3584f4d9f6a8c10438a0eca P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:37.538370 8457 raft_consensus.cc:2258] T 945af26fb3584f4d9f6a8c10438a0eca P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:37.539018 8457 ts_tablet_manager.cc:1875] T 945af26fb3584f4d9f6a8c10438a0eca P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:37.539767 8456 ts_tablet_manager.cc:1888] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:37.539899 8456 log.cc:1196] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/945af26fb3584f4d9f6a8c10438a0eca
I20240502 20:46:37.540241 8456 ts_tablet_manager.cc:1909] T 945af26fb3584f4d9f6a8c10438a0eca P e6b60803d0764d228c863e2da65e3691: Deleting consensus metadata
I20240502 20:46:37.541298 31475 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240502 20:46:37.543148 8457 ts_tablet_manager.cc:1888] T 945af26fb3584f4d9f6a8c10438a0eca P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:37.543274 8457 log.cc:1196] T 945af26fb3584f4d9f6a8c10438a0eca P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/945af26fb3584f4d9f6a8c10438a0eca
I20240502 20:46:37.543321 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet 945af26fb3584f4d9f6a8c10438a0eca (table rebalancer_dynamic_table_new_name [id=f4b6b521d94e400a9111ca8c01764950]) successfully deleted
I20240502 20:46:37.543733 8457 ts_tablet_manager.cc:1909] T 945af26fb3584f4d9f6a8c10438a0eca P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:37.544507 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet 945af26fb3584f4d9f6a8c10438a0eca (table rebalancer_dynamic_table_new_name [id=f4b6b521d94e400a9111ca8c01764950]) successfully deleted
I20240502 20:46:37.546605 29295 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:37.546815 29557 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 20:46:37.550210 31867 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:37.551014 29163 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 20:46:37.556375 29424 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
W20240502 20:46:37.579046 31429 consensus_peers.cc:485] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 20:46:37.580478 31822 consensus_peers.cc:485] T 8feefea4624e49309cef9ecb952bbe96 P e6b60803d0764d228c863e2da65e3691 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:37.581880 31736 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 20:46:37.582760 7256 consensus_queue.cc:1035] T 87670bbef3884cc79a7c7e0533661c12 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 20:46:37.591625 29511 consensus_peers.cc:485] T 87670bbef3884cc79a7c7e0533661c12 P db17f26a73f3447b92aa9a947e3a7057 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:37.599301 31691 consensus_peers.cc:485] T 6436a7d43dab46bca948438f832245ed P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 20:46:37.611574 31429 consensus_peers.cc:485] T 4afde5c1bf754a819f84bcc41212617d P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 20:46:37.626191 31691 consensus_peers.cc:485] T 46c0a5b0870c4a099c7c17ebc0a5d34e P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 20:46:37.628780 29380 consensus_peers.cc:485] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:37.631158 29038 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
W20240502 20:46:37.633417 29380 consensus_peers.cc:485] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:37.639292 31736 tablet_service.cc:1467] Processing CreateTablet for tablet 02f1afd8437046349a7fe341e25be0bc (DEFAULT_TABLE table=rebalancer_dynamic_table [id=f9a9c912d6f048e49254c989784d1000]), partition=
I20240502 20:46:37.639338 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 02f1afd8437046349a7fe341e25be0bc (DEFAULT_TABLE table=rebalancer_dynamic_table [id=f9a9c912d6f048e49254c989784d1000]), partition=
I20240502 20:46:37.639542 31736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02f1afd8437046349a7fe341e25be0bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:37.639575 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02f1afd8437046349a7fe341e25be0bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:37.641314 8472 tablet_bootstrap.cc:492] T 02f1afd8437046349a7fe341e25be0bc P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:37.641455 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 02f1afd8437046349a7fe341e25be0bc (DEFAULT_TABLE table=rebalancer_dynamic_table [id=f9a9c912d6f048e49254c989784d1000]), partition=
I20240502 20:46:37.641665 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02f1afd8437046349a7fe341e25be0bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:37.642504 8473 tablet_bootstrap.cc:492] T 02f1afd8437046349a7fe341e25be0bc P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:37.642596 8472 tablet_bootstrap.cc:654] T 02f1afd8437046349a7fe341e25be0bc P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:37.643322 8473 tablet_bootstrap.cc:654] T 02f1afd8437046349a7fe341e25be0bc P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:37.643640 8472 tablet_bootstrap.cc:492] T 02f1afd8437046349a7fe341e25be0bc P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:37.643718 8472 ts_tablet_manager.cc:1367] T 02f1afd8437046349a7fe341e25be0bc P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 20:46:37.644201 8472 raft_consensus.cc:348] T 02f1afd8437046349a7fe341e25be0bc P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:37.644315 8472 raft_consensus.cc:374] T 02f1afd8437046349a7fe341e25be0bc P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:37.644353 8472 raft_consensus.cc:729] T 02f1afd8437046349a7fe341e25be0bc P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:37.644474 8472 consensus_queue.cc:260] T 02f1afd8437046349a7fe341e25be0bc P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:37.644740 8472 ts_tablet_manager.cc:1398] T 02f1afd8437046349a7fe341e25be0bc P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:37.645745 8476 tablet_bootstrap.cc:492] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
W20240502 20:46:37.646523 31429 consensus_peers.cc:485] T 2a00833b5b404e63b0f720b5c8fb528d P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:37.646852 8476 tablet_bootstrap.cc:654] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:37.648038 8476 tablet_bootstrap.cc:492] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:37.648142 8476 ts_tablet_manager.cc:1367] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:37.648614 8476 raft_consensus.cc:348] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:37.648720 8476 raft_consensus.cc:374] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:37.648754 8476 raft_consensus.cc:729] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:37.648856 8476 consensus_queue.cc:260] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:37.649333 8476 ts_tablet_manager.cc:1398] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:37.655390 8473 tablet_bootstrap.cc:492] T 02f1afd8437046349a7fe341e25be0bc P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:37.655524 8473 ts_tablet_manager.cc:1367] T 02f1afd8437046349a7fe341e25be0bc P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 20:46:37.655884 8473 raft_consensus.cc:348] T 02f1afd8437046349a7fe341e25be0bc P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:37.656041 8473 raft_consensus.cc:374] T 02f1afd8437046349a7fe341e25be0bc P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:37.656093 8473 raft_consensus.cc:729] T 02f1afd8437046349a7fe341e25be0bc P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:37.656241 8473 consensus_queue.cc:260] T 02f1afd8437046349a7fe341e25be0bc P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:37.656514 8473 ts_tablet_manager.cc:1398] T 02f1afd8437046349a7fe341e25be0bc P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20240502 20:46:37.666640 31691 consensus_peers.cc:462] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
W20240502 20:46:37.680351 29511 consensus_peers.cc:485] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 20:46:37.683323 29380 consensus_peers.cc:485] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 20:46:37.688366 29118 consensus_peers.cc:485] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 20:46:37.706331 31691 consensus_peers.cc:485] T 5156cc195fcd4b28b9c7bf1aa8a1409e P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:37.708642 31429 consensus_peers.cc:485] T b718cecdc692408bbeea91bc9fe21fcb P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:37.712513 6850 raft_consensus.cc:482] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:37.712605 6850 raft_consensus.cc:504] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:37.713004 6850 leader_election.cc:288] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:37.713449 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f1afd8437046349a7fe341e25be0bc" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:37.713570 31755 raft_consensus.cc:2454] T 02f1afd8437046349a7fe341e25be0bc P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:37.713747 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f1afd8437046349a7fe341e25be0bc" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:37.713860 29315 raft_consensus.cc:2454] T 02f1afd8437046349a7fe341e25be0bc P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:37.713987 31821 leader_election.cc:302] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e6b60803d0764d228c863e2da65e3691, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:37.714200 6850 raft_consensus.cc:2781] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:37.714251 6850 raft_consensus.cc:482] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:37.714287 6850 raft_consensus.cc:3037] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:37.715422 6850 raft_consensus.cc:504] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:37.715747 6850 leader_election.cc:288] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:37.715978 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f1afd8437046349a7fe341e25be0bc" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:37.716074 29315 raft_consensus.cc:3037] T 02f1afd8437046349a7fe341e25be0bc P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:37.716200 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f1afd8437046349a7fe341e25be0bc" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:37.716300 31755 raft_consensus.cc:3037] T 02f1afd8437046349a7fe341e25be0bc P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:37.717119 29315 raft_consensus.cc:2454] T 02f1afd8437046349a7fe341e25be0bc P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:37.717283 31755 raft_consensus.cc:2454] T 02f1afd8437046349a7fe341e25be0bc P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:37.717625 31821 leader_election.cc:302] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e6b60803d0764d228c863e2da65e3691, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:37.718683 6850 raft_consensus.cc:2781] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:37.718775 6850 raft_consensus.cc:686] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Becoming Leader. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Running, Role: LEADER
I20240502 20:46:37.718916 6850 consensus_queue.cc:237] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:37.721750 29034 catalog_manager.cc:5484] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6b60803d0764d228c863e2da65e3691 (127.22.45.72). New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
W20240502 20:46:37.722479 31822 consensus_peers.cc:485] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 20:46:37.728816 31429 consensus_peers.cc:485] T b699c00ff92b41e9ae24e5982ced3ef9 P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 20:46:37.742014 29249 consensus_peers.cc:485] T f0188e48270242949913034b04dbd3b0 P 09258ac095664fa99caf0707262cde13 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:37.753530 29295 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:37.756991 29163 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 20:46:37.763121 31867 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
W20240502 20:46:37.763845 31822 consensus_peers.cc:485] T 63487637b76a4de89ac3f53c707f074b P e6b60803d0764d228c863e2da65e3691 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:37.767596 31475 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240502 20:46:37.768877 7940 consensus_queue.cc:1035] T 5156cc195fcd4b28b9c7bf1aa8a1409e P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:37.773702 29557 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
W20240502 20:46:37.776225 29118 consensus_peers.cc:485] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:37.783732 29424 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 20:46:37.786597 31736 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
W20240502 20:46:37.806515 29511 consensus_peers.cc:485] T b9c1ec018c544b3bb90fe57fe9995920 P db17f26a73f3447b92aa9a947e3a7057 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 20:46:37.809098 31691 consensus_peers.cc:485] T 1b2fc2ad5cd24a128562714712e89d89 P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:37.811542 8129 consensus_queue.cc:1035] T 0681d19967c74f3da72497366fd9e3fb P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:37.813668 29033 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_64"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:37.815276 7076 consensus_queue.cc:1035] T 0681d19967c74f3da72497366fd9e3fb P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:37.822608 29557 tablet_service.cc:1467] Processing CreateTablet for tablet 127cbf55f7534106a52cbcca3d3dc57f (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=0ccd8634dd30423bba093d28910df668]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:37.822614 31736 tablet_service.cc:1467] Processing CreateTablet for tablet ea26d9d1604e45bc9054e72958573312 (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=0ccd8634dd30423bba093d28910df668]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:37.822883 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 127cbf55f7534106a52cbcca3d3dc57f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:37.822885 31736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea26d9d1604e45bc9054e72958573312. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:37.823567 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 058bb6623be2473ba4cf90c2ca0d52a7 (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=0ccd8634dd30423bba093d28910df668]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:37.823810 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 058bb6623be2473ba4cf90c2ca0d52a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:37.824970 8472 tablet_bootstrap.cc:492] T ea26d9d1604e45bc9054e72958573312 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:37.825909 8476 tablet_bootstrap.cc:492] T 058bb6623be2473ba4cf90c2ca0d52a7 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:37.825925 8472 tablet_bootstrap.cc:654] T ea26d9d1604e45bc9054e72958573312 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:37.825966 29424 tablet_service.cc:1467] Processing CreateTablet for tablet 058bb6623be2473ba4cf90c2ca0d52a7 (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=0ccd8634dd30423bba093d28910df668]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:37.826228 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 058bb6623be2473ba4cf90c2ca0d52a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:37.826238 31474 tablet_service.cc:1467] Processing CreateTablet for tablet 127cbf55f7534106a52cbcca3d3dc57f (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=0ccd8634dd30423bba093d28910df668]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:37.826495 31474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 127cbf55f7534106a52cbcca3d3dc57f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:37.826828 8476 tablet_bootstrap.cc:654] T 058bb6623be2473ba4cf90c2ca0d52a7 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:37.826794 29163 tablet_service.cc:1467] Processing CreateTablet for tablet 127cbf55f7534106a52cbcca3d3dc57f (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=0ccd8634dd30423bba093d28910df668]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:37.827032 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 127cbf55f7534106a52cbcca3d3dc57f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:37.827539 31475 tablet_service.cc:1467] Processing CreateTablet for tablet 058bb6623be2473ba4cf90c2ca0d52a7 (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=0ccd8634dd30423bba093d28910df668]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:37.827771 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 058bb6623be2473ba4cf90c2ca0d52a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:37.828711 29295 tablet_service.cc:1467] Processing CreateTablet for tablet ea26d9d1604e45bc9054e72958573312 (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=0ccd8634dd30423bba093d28910df668]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:37.828953 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea26d9d1604e45bc9054e72958573312. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:37.829463 29162 tablet_service.cc:1467] Processing CreateTablet for tablet ea26d9d1604e45bc9054e72958573312 (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=0ccd8634dd30423bba093d28910df668]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:37.829677 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea26d9d1604e45bc9054e72958573312. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:37.831339 8473 tablet_bootstrap.cc:492] T ea26d9d1604e45bc9054e72958573312 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:37.831974 8526 tablet_bootstrap.cc:492] T 127cbf55f7534106a52cbcca3d3dc57f P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:37.832278 8473 tablet_bootstrap.cc:654] T ea26d9d1604e45bc9054e72958573312 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:37.833379 8472 tablet_bootstrap.cc:492] T ea26d9d1604e45bc9054e72958573312 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:37.833364 29033 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:37.833475 8472 ts_tablet_manager.cc:1367] T ea26d9d1604e45bc9054e72958573312 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.002s
I20240502 20:46:37.833968 8526 tablet_bootstrap.cc:654] T 127cbf55f7534106a52cbcca3d3dc57f P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:37.833935 8472 raft_consensus.cc:348] T ea26d9d1604e45bc9054e72958573312 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:37.834035 8472 raft_consensus.cc:374] T ea26d9d1604e45bc9054e72958573312 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:37.834069 8472 raft_consensus.cc:729] T ea26d9d1604e45bc9054e72958573312 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:37.834178 8472 consensus_queue.cc:260] T ea26d9d1604e45bc9054e72958573312 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:37.834501 8472 ts_tablet_manager.cc:1398] T ea26d9d1604e45bc9054e72958573312 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:37.835830 8531 tablet_bootstrap.cc:492] T 127cbf55f7534106a52cbcca3d3dc57f P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:37.836855 8531 tablet_bootstrap.cc:654] T 127cbf55f7534106a52cbcca3d3dc57f P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:37.837103 8530 tablet_bootstrap.cc:492] T 127cbf55f7534106a52cbcca3d3dc57f P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:37.838888 8530 tablet_bootstrap.cc:654] T 127cbf55f7534106a52cbcca3d3dc57f P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:37.839303 8531 tablet_bootstrap.cc:492] T 127cbf55f7534106a52cbcca3d3dc57f P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:37.839434 8531 ts_tablet_manager.cc:1367] T 127cbf55f7534106a52cbcca3d3dc57f P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:37.839845 8531 raft_consensus.cc:348] T 127cbf55f7534106a52cbcca3d3dc57f P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:37.840579 8531 raft_consensus.cc:374] T 127cbf55f7534106a52cbcca3d3dc57f P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:37.840673 8531 raft_consensus.cc:729] T 127cbf55f7534106a52cbcca3d3dc57f P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:37.840834 8531 consensus_queue.cc:260] T 127cbf55f7534106a52cbcca3d3dc57f P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:37.842280 8531 ts_tablet_manager.cc:1398] T 127cbf55f7534106a52cbcca3d3dc57f P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:37.842438 8531 tablet_bootstrap.cc:492] T ea26d9d1604e45bc9054e72958573312 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:37.842823 29315 raft_consensus.cc:1261] T 02f1afd8437046349a7fe341e25be0bc P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:37.843437 6850 consensus_queue.cc:1035] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:37.844686 8529 tablet_bootstrap.cc:492] T 058bb6623be2473ba4cf90c2ca0d52a7 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:37.845656 8529 tablet_bootstrap.cc:654] T 058bb6623be2473ba4cf90c2ca0d52a7 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:37.845707 31757 raft_consensus.cc:1261] T 02f1afd8437046349a7fe341e25be0bc P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:37.846884 8473 tablet_bootstrap.cc:492] T ea26d9d1604e45bc9054e72958573312 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:37.846971 8473 ts_tablet_manager.cc:1367] T ea26d9d1604e45bc9054e72958573312 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240502 20:46:37.847414 8473 raft_consensus.cc:348] T ea26d9d1604e45bc9054e72958573312 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:37.847515 8473 raft_consensus.cc:374] T ea26d9d1604e45bc9054e72958573312 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:37.847548 8473 raft_consensus.cc:729] T ea26d9d1604e45bc9054e72958573312 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:37.847661 8473 consensus_queue.cc:260] T ea26d9d1604e45bc9054e72958573312 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:37.847730 29034 catalog_manager.cc:5576] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945) does not have the latest schema for tablet 02f1afd8437046349a7fe341e25be0bc (table rebalancer_dynamic_table [id=f9a9c912d6f048e49254c989784d1000]). Expected version 1 got 0
I20240502 20:46:37.847968 8473 ts_tablet_manager.cc:1398] T ea26d9d1604e45bc9054e72958573312 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:37.848129 7076 consensus_queue.cc:1035] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:37.848940 8476 tablet_bootstrap.cc:492] T 058bb6623be2473ba4cf90c2ca0d52a7 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:37.849001 8476 ts_tablet_manager.cc:1367] T 058bb6623be2473ba4cf90c2ca0d52a7 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.000s
I20240502 20:46:37.849411 8476 raft_consensus.cc:348] T 058bb6623be2473ba4cf90c2ca0d52a7 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:37.849519 8476 raft_consensus.cc:374] T 058bb6623be2473ba4cf90c2ca0d52a7 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:37.849553 8476 raft_consensus.cc:729] T 058bb6623be2473ba4cf90c2ca0d52a7 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:37.849655 8476 consensus_queue.cc:260] T 058bb6623be2473ba4cf90c2ca0d52a7 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:37.849952 8476 ts_tablet_manager.cc:1398] T 058bb6623be2473ba4cf90c2ca0d52a7 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:37.852725 8529 tablet_bootstrap.cc:492] T 058bb6623be2473ba4cf90c2ca0d52a7 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:37.852860 8529 ts_tablet_manager.cc:1367] T 058bb6623be2473ba4cf90c2ca0d52a7 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 20:46:37.852926 8531 tablet_bootstrap.cc:654] T ea26d9d1604e45bc9054e72958573312 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:37.853307 29037 catalog_manager.cc:5576] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787) does not have the latest schema for tablet 02f1afd8437046349a7fe341e25be0bc (table rebalancer_dynamic_table [id=f9a9c912d6f048e49254c989784d1000]). Expected version 1 got 0
I20240502 20:46:37.853370 8526 tablet_bootstrap.cc:492] T 127cbf55f7534106a52cbcca3d3dc57f P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:37.853330 8529 raft_consensus.cc:348] T 058bb6623be2473ba4cf90c2ca0d52a7 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:37.853430 8529 raft_consensus.cc:374] T 058bb6623be2473ba4cf90c2ca0d52a7 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:37.853518 8529 raft_consensus.cc:729] T 058bb6623be2473ba4cf90c2ca0d52a7 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:37.853524 8526 ts_tablet_manager.cc:1367] T 127cbf55f7534106a52cbcca3d3dc57f P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20240502 20:46:37.853713 8529 consensus_queue.cc:260] T 058bb6623be2473ba4cf90c2ca0d52a7 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:37.854103 8526 raft_consensus.cc:348] T 127cbf55f7534106a52cbcca3d3dc57f P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:37.854416 8526 raft_consensus.cc:374] T 127cbf55f7534106a52cbcca3d3dc57f P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:37.854598 8526 raft_consensus.cc:729] T 127cbf55f7534106a52cbcca3d3dc57f P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:37.854755 8526 consensus_queue.cc:260] T 127cbf55f7534106a52cbcca3d3dc57f P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:37.855974 8526 ts_tablet_manager.cc:1398] T 127cbf55f7534106a52cbcca3d3dc57f P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:37.856189 8529 ts_tablet_manager.cc:1398] T 058bb6623be2473ba4cf90c2ca0d52a7 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:37.858245 8530 tablet_bootstrap.cc:492] T 127cbf55f7534106a52cbcca3d3dc57f P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:37.858332 8133 tablet.cc:1722] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:37.858577 8530 ts_tablet_manager.cc:1367] T 127cbf55f7534106a52cbcca3d3dc57f P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20240502 20:46:37.858628 8133 tablet.cc:1620] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:37.859474 29574 raft_consensus.cc:1261] T 370bf1bcfb3343c488954ece2488b4c2 P db17f26a73f3447b92aa9a947e3a7057 [term 2 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240502 20:46:37.860417 7239 consensus_queue.cc:1035] T 370bf1bcfb3343c488954ece2488b4c2 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:37.860759 29034 catalog_manager.cc:5576] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945) does not have the latest schema for tablet 02f1afd8437046349a7fe341e25be0bc (table rebalancer_dynamic_table [id=f9a9c912d6f048e49254c989784d1000]). Expected version 1 got 0
I20240502 20:46:37.863270 29034 catalog_manager.cc:5576] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787) does not have the latest schema for tablet 02f1afd8437046349a7fe341e25be0bc (table rebalancer_dynamic_table [id=f9a9c912d6f048e49254c989784d1000]). Expected version 1 got 0
I20240502 20:46:37.863200 8530 raft_consensus.cc:348] T 127cbf55f7534106a52cbcca3d3dc57f P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:37.863498 8530 raft_consensus.cc:374] T 127cbf55f7534106a52cbcca3d3dc57f P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:37.863582 8530 raft_consensus.cc:729] T 127cbf55f7534106a52cbcca3d3dc57f P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:37.863752 8530 consensus_queue.cc:260] T 127cbf55f7534106a52cbcca3d3dc57f P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:37.864956 8530 ts_tablet_manager.cc:1398] T 127cbf55f7534106a52cbcca3d3dc57f P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:37.865159 8530 tablet_bootstrap.cc:492] T 058bb6623be2473ba4cf90c2ca0d52a7 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:37.866173 8530 tablet_bootstrap.cc:654] T 058bb6623be2473ba4cf90c2ca0d52a7 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:37.867430 29037 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=f9a9c912d6f048e49254c989784d1000] alter complete (version 1)
I20240502 20:46:37.867563 8531 tablet_bootstrap.cc:492] T ea26d9d1604e45bc9054e72958573312 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:37.867638 8531 ts_tablet_manager.cc:1367] T ea26d9d1604e45bc9054e72958573312 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.025s user 0.001s sys 0.000s
I20240502 20:46:37.868054 8531 raft_consensus.cc:348] T ea26d9d1604e45bc9054e72958573312 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:37.868211 8531 raft_consensus.cc:374] T ea26d9d1604e45bc9054e72958573312 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:37.868291 8531 raft_consensus.cc:729] T ea26d9d1604e45bc9054e72958573312 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:37.868454 8531 consensus_queue.cc:260] T ea26d9d1604e45bc9054e72958573312 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:37.870862 8531 ts_tablet_manager.cc:1398] T ea26d9d1604e45bc9054e72958573312 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
W20240502 20:46:37.873379 31822 consensus_peers.cc:485] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:37.874634 8545 tablet.cc:1722] T 02f1afd8437046349a7fe341e25be0bc P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:37.874871 8544 tablet.cc:1722] T 02f1afd8437046349a7fe341e25be0bc P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:37.875514 8530 tablet_bootstrap.cc:492] T 058bb6623be2473ba4cf90c2ca0d52a7 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:37.875638 8530 ts_tablet_manager.cc:1367] T 058bb6623be2473ba4cf90c2ca0d52a7 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 20:46:37.876199 8530 raft_consensus.cc:348] T 058bb6623be2473ba4cf90c2ca0d52a7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:37.876299 8530 raft_consensus.cc:374] T 058bb6623be2473ba4cf90c2ca0d52a7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:37.876327 8530 raft_consensus.cc:729] T 058bb6623be2473ba4cf90c2ca0d52a7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:37.876497 8530 consensus_queue.cc:260] T 058bb6623be2473ba4cf90c2ca0d52a7 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:37.877496 8530 ts_tablet_manager.cc:1398] T 058bb6623be2473ba4cf90c2ca0d52a7 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:37.877861 8545 tablet.cc:1620] T 02f1afd8437046349a7fe341e25be0bc P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:37.878144 8544 tablet.cc:1620] T 02f1afd8437046349a7fe341e25be0bc P 09258ac095664fa99caf0707262cde13: MemRowSet was empty: no flush needed.
W20240502 20:46:37.917655 31822 consensus_peers.cc:485] T 7a8b04cb606f423f8dd1d489d47408d2 P e6b60803d0764d228c863e2da65e3691 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:37.923100 29314 raft_consensus.cc:1261] T 370bf1bcfb3343c488954ece2488b4c2 P 09258ac095664fa99caf0707262cde13 [term 2 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240502 20:46:37.923507 7239 consensus_queue.cc:1035] T 370bf1bcfb3343c488954ece2488b4c2 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:37.931201 31757 raft_consensus.cc:1261] T 370bf1bcfb3343c488954ece2488b4c2 P e7ab6b9053664ee5aa71c7978e26fe6a [term 2 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240502 20:46:37.938099 7979 consensus_queue.cc:1035] T 370bf1bcfb3343c488954ece2488b4c2 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:37.971403 29040 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
W20240502 20:46:37.979617 29027 catalog_manager.cc:4573] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1e16a4014cf24e84abc7a7d42460dea3: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:37.992903 8133 tablet.cc:1722] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:37.993214 8133 tablet.cc:1620] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:37.993862 8544 tablet.cc:1722] T 02f1afd8437046349a7fe341e25be0bc P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:37.994119 8544 tablet.cc:1620] T 02f1afd8437046349a7fe341e25be0bc P 09258ac095664fa99caf0707262cde13: MemRowSet was empty: no flush needed.
I20240502 20:46:37.998193 8545 tablet.cc:1722] T 02f1afd8437046349a7fe341e25be0bc P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:37.998510 8545 tablet.cc:1620] T 02f1afd8437046349a7fe341e25be0bc P e7ab6b9053664ee5aa71c7978e26fe6a: MemRowSet was empty: no flush needed.
I20240502 20:46:38.003957 29042 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=f9a9c912d6f048e49254c989784d1000] alter complete (version 2)
W20240502 20:46:38.004837 29511 consensus_peers.cc:485] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:38.045481 7256 raft_consensus.cc:482] T 127cbf55f7534106a52cbcca3d3dc57f P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:38.045588 7256 raft_consensus.cc:504] T 127cbf55f7534106a52cbcca3d3dc57f P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:38.045984 7256 leader_election.cc:288] T 127cbf55f7534106a52cbcca3d3dc57f P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:38.046339 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "127cbf55f7534106a52cbcca3d3dc57f" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:38.046327 31493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "127cbf55f7534106a52cbcca3d3dc57f" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:38.046481 29175 raft_consensus.cc:2454] T 127cbf55f7534106a52cbcca3d3dc57f P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:38.046500 31493 raft_consensus.cc:2454] T 127cbf55f7534106a52cbcca3d3dc57f P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:38.046778 29512 leader_election.cc:302] T 127cbf55f7534106a52cbcca3d3dc57f P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:38.046998 7256 raft_consensus.cc:2781] T 127cbf55f7534106a52cbcca3d3dc57f P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:38.047053 7256 raft_consensus.cc:482] T 127cbf55f7534106a52cbcca3d3dc57f P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:38.047078 7256 raft_consensus.cc:3037] T 127cbf55f7534106a52cbcca3d3dc57f P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:38.047857 7256 raft_consensus.cc:504] T 127cbf55f7534106a52cbcca3d3dc57f P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:38.048183 7256 leader_election.cc:288] T 127cbf55f7534106a52cbcca3d3dc57f P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:38.048445 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "127cbf55f7534106a52cbcca3d3dc57f" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:38.048451 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "127cbf55f7534106a52cbcca3d3dc57f" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:38.048552 31495 raft_consensus.cc:3037] T 127cbf55f7534106a52cbcca3d3dc57f P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:38.048555 29175 raft_consensus.cc:3037] T 127cbf55f7534106a52cbcca3d3dc57f P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:38.049280 31495 raft_consensus.cc:2454] T 127cbf55f7534106a52cbcca3d3dc57f P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:38.049376 29175 raft_consensus.cc:2454] T 127cbf55f7534106a52cbcca3d3dc57f P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:38.049532 29512 leader_election.cc:302] T 127cbf55f7534106a52cbcca3d3dc57f P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:38.049706 7256 raft_consensus.cc:2781] T 127cbf55f7534106a52cbcca3d3dc57f P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:38.049806 7256 raft_consensus.cc:686] T 127cbf55f7534106a52cbcca3d3dc57f P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Becoming Leader. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Running, Role: LEADER
I20240502 20:46:38.049949 7256 consensus_queue.cc:237] T 127cbf55f7534106a52cbcca3d3dc57f P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:38.052039 29033 catalog_manager.cc:5484] T 127cbf55f7534106a52cbcca3d3dc57f P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: term changed from 0 to 1, leader changed from <none> to db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:38.107116 29035 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:38.111907 7979 raft_consensus.cc:482] T ea26d9d1604e45bc9054e72958573312 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:38.111856 8133 tablet.cc:1722] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:38.112006 7979 raft_consensus.cc:504] T ea26d9d1604e45bc9054e72958573312 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:38.112435 7979 leader_election.cc:288] T ea26d9d1604e45bc9054e72958573312 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:38.112676 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea26d9d1604e45bc9054e72958573312" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:38.112867 29314 raft_consensus.cc:2454] T ea26d9d1604e45bc9054e72958573312 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:38.112862 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea26d9d1604e45bc9054e72958573312" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:38.113124 29119 leader_election.cc:302] T ea26d9d1604e45bc9054e72958573312 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:38.113148 31757 raft_consensus.cc:2454] T ea26d9d1604e45bc9054e72958573312 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:38.113322 7979 raft_consensus.cc:2781] T ea26d9d1604e45bc9054e72958573312 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:38.113390 7979 raft_consensus.cc:482] T ea26d9d1604e45bc9054e72958573312 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:38.113436 7979 raft_consensus.cc:3037] T ea26d9d1604e45bc9054e72958573312 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:38.114853 7979 raft_consensus.cc:504] T ea26d9d1604e45bc9054e72958573312 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:38.115491 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea26d9d1604e45bc9054e72958573312" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:38.115594 31755 raft_consensus.cc:3037] T ea26d9d1604e45bc9054e72958573312 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:38.115854 8545 tablet.cc:1722] T 02f1afd8437046349a7fe341e25be0bc P e7ab6b9053664ee5aa71c7978e26fe6a: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:38.116290 8544 tablet.cc:1722] T 02f1afd8437046349a7fe341e25be0bc P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:38.116564 31755 raft_consensus.cc:2454] T ea26d9d1604e45bc9054e72958573312 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:38.117990 29117 leader_election.cc:302] T ea26d9d1604e45bc9054e72958573312 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:38.118153 8588 raft_consensus.cc:2781] T ea26d9d1604e45bc9054e72958573312 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:38.118232 8588 raft_consensus.cc:686] T ea26d9d1604e45bc9054e72958573312 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:38.118367 8588 consensus_queue.cc:237] T ea26d9d1604e45bc9054e72958573312 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:38.120625 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea26d9d1604e45bc9054e72958573312" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:38.120780 29314 raft_consensus.cc:3037] T ea26d9d1604e45bc9054e72958573312 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:38.120857 29038 catalog_manager.cc:5484] T ea26d9d1604e45bc9054e72958573312 P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:38.121094 7979 leader_election.cc:288] T ea26d9d1604e45bc9054e72958573312 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:38.122090 29314 raft_consensus.cc:2454] T ea26d9d1604e45bc9054e72958573312 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:38.122581 7829 raft_consensus.cc:482] T 058bb6623be2473ba4cf90c2ca0d52a7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:38.122671 7829 raft_consensus.cc:504] T 058bb6623be2473ba4cf90c2ca0d52a7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:38.123113 7829 leader_election.cc:288] T 058bb6623be2473ba4cf90c2ca0d52a7 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:38.123291 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "058bb6623be2473ba4cf90c2ca0d52a7" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:38.123432 29446 raft_consensus.cc:2454] T 058bb6623be2473ba4cf90c2ca0d52a7 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:38.123694 31430 leader_election.cc:302] T 058bb6623be2473ba4cf90c2ca0d52a7 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:38.123765 31886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "058bb6623be2473ba4cf90c2ca0d52a7" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:38.123874 31886 raft_consensus.cc:2454] T 058bb6623be2473ba4cf90c2ca0d52a7 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 0.
I20240502 20:46:38.123939 7829 raft_consensus.cc:2781] T 058bb6623be2473ba4cf90c2ca0d52a7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:38.123980 7829 raft_consensus.cc:482] T 058bb6623be2473ba4cf90c2ca0d52a7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:38.124013 7829 raft_consensus.cc:3037] T 058bb6623be2473ba4cf90c2ca0d52a7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
W20240502 20:46:38.124533 29380 consensus_peers.cc:485] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:38.124598 29033 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=f9a9c912d6f048e49254c989784d1000] alter complete (version 3)
I20240502 20:46:38.125027 7829 raft_consensus.cc:504] T 058bb6623be2473ba4cf90c2ca0d52a7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:38.125633 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "058bb6623be2473ba4cf90c2ca0d52a7" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:38.125684 31886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "058bb6623be2473ba4cf90c2ca0d52a7" candidate_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:38.125387 7829 leader_election.cc:288] T 058bb6623be2473ba4cf90c2ca0d52a7 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Requested vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:38.125746 29446 raft_consensus.cc:3037] T 058bb6623be2473ba4cf90c2ca0d52a7 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:38.125782 31886 raft_consensus.cc:3037] T 058bb6623be2473ba4cf90c2ca0d52a7 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:38.126617 31886 raft_consensus.cc:2454] T 058bb6623be2473ba4cf90c2ca0d52a7 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:38.126713 29446 raft_consensus.cc:2454] T 058bb6623be2473ba4cf90c2ca0d52a7 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9c1af8cf814030bde56b67ca5c2fbe in term 1.
I20240502 20:46:38.126866 31429 leader_election.cc:302] T 058bb6623be2473ba4cf90c2ca0d52a7 P 2a9c1af8cf814030bde56b67ca5c2fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:38.127022 7829 raft_consensus.cc:2781] T 058bb6623be2473ba4cf90c2ca0d52a7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:38.127115 7829 raft_consensus.cc:686] T 058bb6623be2473ba4cf90c2ca0d52a7 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 LEADER]: Becoming Leader. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Running, Role: LEADER
I20240502 20:46:38.127234 7829 consensus_queue.cc:237] T 058bb6623be2473ba4cf90c2ca0d52a7 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:38.129210 29033 catalog_manager.cc:5484] T 058bb6623be2473ba4cf90c2ca0d52a7 P 2a9c1af8cf814030bde56b67ca5c2fbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69). New cstate: current_term: 1 leader_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:38.159914 29162 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 20:46:38.164763 31736 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 20:46:38.166729 29295 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:38.168857 29557 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 20:46:38.170153 29424 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 20:46:38.179713 31475 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240502 20:46:38.196802 31867 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 20:46:38.227821 29033 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:38.227972 29033 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:38.230170 29033 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 02f1afd8437046349a7fe341e25be0bc
I20240502 20:46:38.230649 31867 tablet_service.cc:1514] Processing DeleteTablet for tablet 02f1afd8437046349a7fe341e25be0bc with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:38 UTC) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:38.230757 29295 tablet_service.cc:1514] Processing DeleteTablet for tablet 02f1afd8437046349a7fe341e25be0bc with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:38 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:38.230911 8604 tablet_replica.cc:330] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:38.230983 31736 tablet_service.cc:1514] Processing DeleteTablet for tablet 02f1afd8437046349a7fe341e25be0bc with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:38 UTC) from {username='slave'} at 127.0.0.1:33864
I20240502 20:46:38.231050 8604 raft_consensus.cc:2229] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:38.231232 8605 tablet_replica.cc:330] T 02f1afd8437046349a7fe341e25be0bc P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:38.231242 8604 raft_consensus.cc:2258] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:38.231349 8606 tablet_replica.cc:330] T 02f1afd8437046349a7fe341e25be0bc P e7ab6b9053664ee5aa71c7978e26fe6a: stopping tablet replica
I20240502 20:46:38.231429 8605 raft_consensus.cc:2229] T 02f1afd8437046349a7fe341e25be0bc P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:38.231496 8606 raft_consensus.cc:2229] T 02f1afd8437046349a7fe341e25be0bc P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:38.231580 8605 raft_consensus.cc:2258] T 02f1afd8437046349a7fe341e25be0bc P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:38.231619 8606 raft_consensus.cc:2258] T 02f1afd8437046349a7fe341e25be0bc P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:38.232306 8606 ts_tablet_manager.cc:1875] T 02f1afd8437046349a7fe341e25be0bc P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:38.233364 8605 ts_tablet_manager.cc:1875] T 02f1afd8437046349a7fe341e25be0bc P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:38.233585 8604 ts_tablet_manager.cc:1875] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:38.234849 8606 ts_tablet_manager.cc:1888] T 02f1afd8437046349a7fe341e25be0bc P e7ab6b9053664ee5aa71c7978e26fe6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:38.234944 8606 log.cc:1196] T 02f1afd8437046349a7fe341e25be0bc P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/wals/02f1afd8437046349a7fe341e25be0bc
I20240502 20:46:38.235205 8606 ts_tablet_manager.cc:1909] T 02f1afd8437046349a7fe341e25be0bc P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting consensus metadata
I20240502 20:46:38.235935 29026 catalog_manager.cc:4830] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): tablet 02f1afd8437046349a7fe341e25be0bc (table rebalancer_dynamic_table_new_name [id=f9a9c912d6f048e49254c989784d1000]) successfully deleted
I20240502 20:46:38.235699 8604 ts_tablet_manager.cc:1888] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:38.236047 8604 log.cc:1196] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/02f1afd8437046349a7fe341e25be0bc
I20240502 20:46:38.236299 8604 ts_tablet_manager.cc:1909] T 02f1afd8437046349a7fe341e25be0bc P e6b60803d0764d228c863e2da65e3691: Deleting consensus metadata
I20240502 20:46:38.236897 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet 02f1afd8437046349a7fe341e25be0bc (table rebalancer_dynamic_table_new_name [id=f9a9c912d6f048e49254c989784d1000]) successfully deleted
I20240502 20:46:38.240659 8605 ts_tablet_manager.cc:1888] T 02f1afd8437046349a7fe341e25be0bc P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:38.240725 8605 log.cc:1196] T 02f1afd8437046349a7fe341e25be0bc P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/02f1afd8437046349a7fe341e25be0bc
I20240502 20:46:38.241035 8605 ts_tablet_manager.cc:1909] T 02f1afd8437046349a7fe341e25be0bc P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:38.241595 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet 02f1afd8437046349a7fe341e25be0bc (table rebalancer_dynamic_table_new_name [id=f9a9c912d6f048e49254c989784d1000]) successfully deleted
W20240502 20:46:38.264320 31691 consensus_peers.cc:462] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:38.332533 29036 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:38.338698 31475 tablet_service.cc:1467] Processing CreateTablet for tablet 7cb63dbbd86c46c38e958e65af9c2c84 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=859757995ff74518b2021e2c01d75272]), partition=
I20240502 20:46:38.338961 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cb63dbbd86c46c38e958e65af9c2c84. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:38.341240 8530 tablet_bootstrap.cc:492] T 7cb63dbbd86c46c38e958e65af9c2c84 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:38.341390 29424 tablet_service.cc:1467] Processing CreateTablet for tablet 7cb63dbbd86c46c38e958e65af9c2c84 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=859757995ff74518b2021e2c01d75272]), partition=
I20240502 20:46:38.341572 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cb63dbbd86c46c38e958e65af9c2c84. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:38.342094 8530 tablet_bootstrap.cc:654] T 7cb63dbbd86c46c38e958e65af9c2c84 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:38.342854 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 7cb63dbbd86c46c38e958e65af9c2c84 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=859757995ff74518b2021e2c01d75272]), partition=
I20240502 20:46:38.343084 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cb63dbbd86c46c38e958e65af9c2c84. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:38.343725 8529 tablet_bootstrap.cc:492] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:38.344529 8529 tablet_bootstrap.cc:654] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:38.345453 8476 tablet_bootstrap.cc:492] T 7cb63dbbd86c46c38e958e65af9c2c84 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:38.346256 8476 tablet_bootstrap.cc:654] T 7cb63dbbd86c46c38e958e65af9c2c84 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:38.350725 8476 tablet_bootstrap.cc:492] T 7cb63dbbd86c46c38e958e65af9c2c84 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:38.350800 8476 ts_tablet_manager.cc:1367] T 7cb63dbbd86c46c38e958e65af9c2c84 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 20:46:38.351218 8476 raft_consensus.cc:348] T 7cb63dbbd86c46c38e958e65af9c2c84 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:38.351315 8476 raft_consensus.cc:374] T 7cb63dbbd86c46c38e958e65af9c2c84 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:38.351342 8476 raft_consensus.cc:729] T 7cb63dbbd86c46c38e958e65af9c2c84 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:38.351444 8476 consensus_queue.cc:260] T 7cb63dbbd86c46c38e958e65af9c2c84 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:38.351702 8476 ts_tablet_manager.cc:1398] T 7cb63dbbd86c46c38e958e65af9c2c84 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:38.352562 8530 tablet_bootstrap.cc:492] T 7cb63dbbd86c46c38e958e65af9c2c84 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:38.352640 8530 ts_tablet_manager.cc:1367] T 7cb63dbbd86c46c38e958e65af9c2c84 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 20:46:38.353049 8530 raft_consensus.cc:348] T 7cb63dbbd86c46c38e958e65af9c2c84 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:38.353133 8530 raft_consensus.cc:374] T 7cb63dbbd86c46c38e958e65af9c2c84 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:38.353159 8530 raft_consensus.cc:729] T 7cb63dbbd86c46c38e958e65af9c2c84 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:38.353272 8530 consensus_queue.cc:260] T 7cb63dbbd86c46c38e958e65af9c2c84 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:38.353508 8530 ts_tablet_manager.cc:1398] T 7cb63dbbd86c46c38e958e65af9c2c84 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:38.358063 8529 tablet_bootstrap.cc:492] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:38.358155 8529 ts_tablet_manager.cc:1367] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 20:46:38.358633 8529 raft_consensus.cc:348] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:38.358726 8529 raft_consensus.cc:374] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:38.358750 8529 raft_consensus.cc:729] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:38.358855 8529 consensus_queue.cc:260] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:38.359128 8529 ts_tablet_manager.cc:1398] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:38.363936 7256 raft_consensus.cc:482] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:38.364027 7256 raft_consensus.cc:504] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:38.364393 7256 leader_election.cc:288] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:38.364923 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "378b1151c07648919504bacdcb20a087" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:38.365029 29446 raft_consensus.cc:2454] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 2.
I20240502 20:46:38.365844 29512 leader_election.cc:302] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:38.370144 7256 raft_consensus.cc:2781] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20240502 20:46:38.370227 7256 raft_consensus.cc:482] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:38.370265 7256 raft_consensus.cc:3037] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:38.371265 7256 raft_consensus.cc:504] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:38.371596 7256 leader_election.cc:288] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 3 election: Requested vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751)
I20240502 20:46:38.372160 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "378b1151c07648919504bacdcb20a087" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:38.372285 29446 raft_consensus.cc:3037] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:38.373296 29446 raft_consensus.cc:2454] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 3.
I20240502 20:46:38.375463 29512 leader_election.cc:302] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:38.375742 5346 raft_consensus.cc:2781] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 20:46:38.375814 5346 raft_consensus.cc:686] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [term 3 LEADER]: Becoming Leader. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Running, Role: LEADER
I20240502 20:46:38.375921 5346 consensus_queue.cc:237] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } }
I20240502 20:46:38.376102 29162 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
W20240502 20:46:38.379937 29511 leader_election.cc:334] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
W20240502 20:46:38.380960 29511 leader_election.cc:334] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:38.382803 29033 catalog_manager.cc:5484] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: term changed from 1 to 3, leader changed from 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70) to db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68). New cstate: current_term: 3 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "315915efcf2b413ca7f9d874338e1b2d" member_type: VOTER last_known_addr { host: "127.22.45.70" port: 46751 } } }
I20240502 20:46:38.388026 29295 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:38.394660 31736 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 20:46:38.399309 31867 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:38.403060 29424 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 20:46:38.405745 29557 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 20:46:38.418524 31475 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240502 20:46:38.478746 31757 consensus_queue.cc:237] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:38.480433 31886 raft_consensus.cc:1261] T 75ace61b6538462b9bd5c0820eeacca9 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer e7ab6b9053664ee5aa71c7978e26fe6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:38.480444 29446 raft_consensus.cc:1261] T 75ace61b6538462b9bd5c0820eeacca9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer e7ab6b9053664ee5aa71c7978e26fe6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:38.480835 8264 consensus_queue.cc:1035] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:38.481249 8680 consensus_queue.cc:1035] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:38.482664 8264 raft_consensus.cc:2932] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:38.483214 31886 raft_consensus.cc:2932] T 75ace61b6538462b9bd5c0820eeacca9 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:38.483490 29446 raft_consensus.cc:2932] T 75ace61b6538462b9bd5c0820eeacca9 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:38.486363 29033 catalog_manager.cc:5484] T 75ace61b6538462b9bd5c0820eeacca9 P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: config changed from index 2 to 3, VOTER 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69) evicted. New cstate: current_term: 1 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
I20240502 20:46:38.490576 31475 tablet_service.cc:1514] Processing DeleteTablet for tablet 75ace61b6538462b9bd5c0820eeacca9 with delete_type TABLET_DATA_TOMBSTONED (TS 2a9c1af8cf814030bde56b67ca5c2fbe not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:40276
I20240502 20:46:38.490906 8688 tablet_replica.cc:330] T 75ace61b6538462b9bd5c0820eeacca9 P 2a9c1af8cf814030bde56b67ca5c2fbe: stopping tablet replica
I20240502 20:46:38.491058 8688 raft_consensus.cc:2229] T 75ace61b6538462b9bd5c0820eeacca9 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:38.491235 8688 raft_consensus.cc:2258] T 75ace61b6538462b9bd5c0820eeacca9 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:38.491640 8688 ts_tablet_manager.cc:1875] T 75ace61b6538462b9bd5c0820eeacca9 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:38.494136 8688 ts_tablet_manager.cc:1888] T 75ace61b6538462b9bd5c0820eeacca9 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 20:46:38.494246 8688 log.cc:1196] T 75ace61b6538462b9bd5c0820eeacca9 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/wals/75ace61b6538462b9bd5c0820eeacca9
I20240502 20:46:38.495055 29028 catalog_manager.cc:4830] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): tablet 75ace61b6538462b9bd5c0820eeacca9 (table rebalancer_extra_table_30 [id=ff32eaad49184ef9bdfa32309107f0ac]) successfully deleted
I20240502 20:46:38.507571 6234 raft_consensus.cc:482] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:38.507664 6234 raft_consensus.cc:504] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:38.508008 6234 leader_election.cc:288] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:38.508270 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cb63dbbd86c46c38e958e65af9c2c84" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:38.508383 31495 raft_consensus.cc:2454] T 7cb63dbbd86c46c38e958e65af9c2c84 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:38.508354 31886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cb63dbbd86c46c38e958e65af9c2c84" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:38.508488 31886 raft_consensus.cc:2454] T 7cb63dbbd86c46c38e958e65af9c2c84 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:38.508648 29381 leader_election.cc:302] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:38.508847 6234 raft_consensus.cc:2781] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:38.508914 6234 raft_consensus.cc:482] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:38.508951 6234 raft_consensus.cc:3037] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:38.509853 6234 raft_consensus.cc:504] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:38.510203 6234 leader_election.cc:288] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:38.510491 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cb63dbbd86c46c38e958e65af9c2c84" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:38.510593 31495 raft_consensus.cc:3037] T 7cb63dbbd86c46c38e958e65af9c2c84 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:38.510562 31886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cb63dbbd86c46c38e958e65af9c2c84" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:38.510663 31886 raft_consensus.cc:3037] T 7cb63dbbd86c46c38e958e65af9c2c84 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:38.511443 31495 raft_consensus.cc:2454] T 7cb63dbbd86c46c38e958e65af9c2c84 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:38.511457 31886 raft_consensus.cc:2454] T 7cb63dbbd86c46c38e958e65af9c2c84 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:38.511701 29380 leader_election.cc:302] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dbcdc6ac635b44b185c3eba63f66e9d6, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:38.511960 6234 raft_consensus.cc:2781] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:38.512037 6234 raft_consensus.cc:686] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:38.512157 6234 consensus_queue.cc:237] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:38.514446 29038 catalog_manager.cc:5484] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:38.520926 7256 consensus_queue.cc:1035] T 127cbf55f7534106a52cbcca3d3dc57f P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:38.524287 7256 consensus_queue.cc:1035] T 127cbf55f7534106a52cbcca3d3dc57f P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:38.600533 8588 consensus_queue.cc:1035] T ea26d9d1604e45bc9054e72958573312 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:38.610749 7979 consensus_queue.cc:1035] T ea26d9d1604e45bc9054e72958573312 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:38.613490 29557 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 20:46:38.615360 29162 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 20:46:38.617583 29295 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:38.622000 31736 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 20:46:38.647538 31475 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240502 20:46:38.648821 31867 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:38.657191 29034 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:38.657291 29037 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_66"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:38.666430 31475 tablet_service.cc:1467] Processing CreateTablet for tablet ee6f512ddcb64018b0bbae67c9b3854b (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=09828e72e84f44ceacbd1eb85446a7e4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:38.666723 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee6f512ddcb64018b0bbae67c9b3854b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:38.667038 31736 tablet_service.cc:1467] Processing CreateTablet for tablet 5f7707f47e43472dbaf966db41f0745a (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=09828e72e84f44ceacbd1eb85446a7e4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:38.667304 31736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f7707f47e43472dbaf966db41f0745a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:38.669080 29294 tablet_service.cc:1467] Processing CreateTablet for tablet 30e7bac7fe5f4cd3895ab29e9fc5986b (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=09828e72e84f44ceacbd1eb85446a7e4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:38.669277 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30e7bac7fe5f4cd3895ab29e9fc5986b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:38.669477 8748 tablet_bootstrap.cc:492] T 5f7707f47e43472dbaf966db41f0745a P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:38.669740 31735 tablet_service.cc:1467] Processing CreateTablet for tablet ee6f512ddcb64018b0bbae67c9b3854b (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=09828e72e84f44ceacbd1eb85446a7e4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:38.669936 31735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee6f512ddcb64018b0bbae67c9b3854b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:38.670629 8748 tablet_bootstrap.cc:654] T 5f7707f47e43472dbaf966db41f0745a P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:38.671448 8750 tablet_bootstrap.cc:492] T 30e7bac7fe5f4cd3895ab29e9fc5986b P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:38.671741 8530 tablet_bootstrap.cc:492] T ee6f512ddcb64018b0bbae67c9b3854b P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:38.672019 31474 tablet_service.cc:1467] Processing CreateTablet for tablet 30e7bac7fe5f4cd3895ab29e9fc5986b (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=09828e72e84f44ceacbd1eb85446a7e4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:38.672255 31474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30e7bac7fe5f4cd3895ab29e9fc5986b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:38.672456 8750 tablet_bootstrap.cc:654] T 30e7bac7fe5f4cd3895ab29e9fc5986b P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:38.672564 8530 tablet_bootstrap.cc:654] T ee6f512ddcb64018b0bbae67c9b3854b P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:38.673763 8530 tablet_bootstrap.cc:492] T ee6f512ddcb64018b0bbae67c9b3854b P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:38.673846 8530 ts_tablet_manager.cc:1367] T ee6f512ddcb64018b0bbae67c9b3854b P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:38.674310 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 5f7707f47e43472dbaf966db41f0745a (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=09828e72e84f44ceacbd1eb85446a7e4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:38.674566 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f7707f47e43472dbaf966db41f0745a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:38.674641 8530 raft_consensus.cc:348] T ee6f512ddcb64018b0bbae67c9b3854b P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:38.674732 8530 raft_consensus.cc:374] T ee6f512ddcb64018b0bbae67c9b3854b P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:38.674762 8530 raft_consensus.cc:729] T ee6f512ddcb64018b0bbae67c9b3854b P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:38.674872 8530 consensus_queue.cc:260] T ee6f512ddcb64018b0bbae67c9b3854b P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:38.674923 29162 tablet_service.cc:1467] Processing CreateTablet for tablet ee6f512ddcb64018b0bbae67c9b3854b (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=09828e72e84f44ceacbd1eb85446a7e4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:38.675135 8530 ts_tablet_manager.cc:1398] T ee6f512ddcb64018b0bbae67c9b3854b P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:38.675179 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee6f512ddcb64018b0bbae67c9b3854b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:38.675287 8530 tablet_bootstrap.cc:492] T 30e7bac7fe5f4cd3895ab29e9fc5986b P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:38.675426 31867 tablet_service.cc:1467] Processing CreateTablet for tablet 5f7707f47e43472dbaf966db41f0745a (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=09828e72e84f44ceacbd1eb85446a7e4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:38.675657 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f7707f47e43472dbaf966db41f0745a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:38.676142 8530 tablet_bootstrap.cc:654] T 30e7bac7fe5f4cd3895ab29e9fc5986b P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:38.679913 29424 tablet_service.cc:1467] Processing CreateTablet for tablet 30e7bac7fe5f4cd3895ab29e9fc5986b (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=09828e72e84f44ceacbd1eb85446a7e4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:38.680905 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30e7bac7fe5f4cd3895ab29e9fc5986b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:38.686394 7829 consensus_queue.cc:1035] T 058bb6623be2473ba4cf90c2ca0d52a7 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:38.686590 8529 tablet_bootstrap.cc:492] T 30e7bac7fe5f4cd3895ab29e9fc5986b P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:38.687498 8529 tablet_bootstrap.cc:654] T 30e7bac7fe5f4cd3895ab29e9fc5986b P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:38.688958 8476 tablet_bootstrap.cc:492] T 5f7707f47e43472dbaf966db41f0745a P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:38.689316 31886 raft_consensus.cc:1261] T 7cb63dbbd86c46c38e958e65af9c2c84 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:38.689824 8476 tablet_bootstrap.cc:654] T 5f7707f47e43472dbaf966db41f0745a P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:38.690097 7116 consensus_queue.cc:1035] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:38.690181 31495 raft_consensus.cc:1261] T 7cb63dbbd86c46c38e958e65af9c2c84 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Refusing update from remote peer dbcdc6ac635b44b185c3eba63f66e9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:38.690425 8530 tablet_bootstrap.cc:492] T 30e7bac7fe5f4cd3895ab29e9fc5986b P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:38.690531 8530 ts_tablet_manager.cc:1367] T 30e7bac7fe5f4cd3895ab29e9fc5986b P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.000s
I20240502 20:46:38.690984 8530 raft_consensus.cc:348] T 30e7bac7fe5f4cd3895ab29e9fc5986b P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:38.691080 8530 raft_consensus.cc:374] T 30e7bac7fe5f4cd3895ab29e9fc5986b P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:38.691110 8530 raft_consensus.cc:729] T 30e7bac7fe5f4cd3895ab29e9fc5986b P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:38.691202 8530 consensus_queue.cc:260] T 30e7bac7fe5f4cd3895ab29e9fc5986b P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:38.691469 8530 ts_tablet_manager.cc:1398] T 30e7bac7fe5f4cd3895ab29e9fc5986b P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:38.692093 6234 consensus_queue.cc:1035] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:38.692387 8750 tablet_bootstrap.cc:492] T 30e7bac7fe5f4cd3895ab29e9fc5986b P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:38.692487 8750 ts_tablet_manager.cc:1367] T 30e7bac7fe5f4cd3895ab29e9fc5986b P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20240502 20:46:38.692982 8750 raft_consensus.cc:348] T 30e7bac7fe5f4cd3895ab29e9fc5986b P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:38.693089 8750 raft_consensus.cc:374] T 30e7bac7fe5f4cd3895ab29e9fc5986b P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:38.693121 8750 raft_consensus.cc:729] T 30e7bac7fe5f4cd3895ab29e9fc5986b P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:38.693229 8750 consensus_queue.cc:260] T 30e7bac7fe5f4cd3895ab29e9fc5986b P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:38.693509 8750 ts_tablet_manager.cc:1398] T 30e7bac7fe5f4cd3895ab29e9fc5986b P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:38.693640 8750 tablet_bootstrap.cc:492] T 5f7707f47e43472dbaf966db41f0745a P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:38.694515 8750 tablet_bootstrap.cc:654] T 5f7707f47e43472dbaf966db41f0745a P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:38.697664 29034 catalog_manager.cc:5576] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207) does not have the latest schema for tablet 7cb63dbbd86c46c38e958e65af9c2c84 (table rebalancer_dynamic_table [id=859757995ff74518b2021e2c01d75272]). Expected version 1 got 0
I20240502 20:46:38.698207 29424 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 20:46:38.700446 29034 catalog_manager.cc:5576] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887) does not have the latest schema for tablet 7cb63dbbd86c46c38e958e65af9c2c84 (table rebalancer_dynamic_table [id=859757995ff74518b2021e2c01d75272]). Expected version 1 got 0
I20240502 20:46:38.704819 8748 tablet_bootstrap.cc:492] T 5f7707f47e43472dbaf966db41f0745a P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:38.704902 8748 ts_tablet_manager.cc:1367] T 5f7707f47e43472dbaf966db41f0745a P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.035s user 0.002s sys 0.000s
I20240502 20:46:38.705097 8765 tablet.cc:1722] T 7cb63dbbd86c46c38e958e65af9c2c84 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:38.705302 8748 raft_consensus.cc:348] T 5f7707f47e43472dbaf966db41f0745a P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:38.705391 8748 raft_consensus.cc:374] T 5f7707f47e43472dbaf966db41f0745a P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:38.705426 8748 raft_consensus.cc:729] T 5f7707f47e43472dbaf966db41f0745a P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:38.705509 8765 tablet.cc:1620] T 7cb63dbbd86c46c38e958e65af9c2c84 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:38.705531 8748 consensus_queue.cc:260] T 5f7707f47e43472dbaf966db41f0745a P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:38.705792 8748 ts_tablet_manager.cc:1398] T 5f7707f47e43472dbaf966db41f0745a P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:38.706336 8748 tablet_bootstrap.cc:492] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:38.707207 8748 tablet_bootstrap.cc:654] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:38.708236 8748 tablet_bootstrap.cc:492] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:38.708318 8748 ts_tablet_manager.cc:1367] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:38.708720 8748 raft_consensus.cc:348] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:38.708814 8748 raft_consensus.cc:374] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:38.708848 8748 raft_consensus.cc:729] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:38.708940 8748 consensus_queue.cc:260] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:38.709175 8748 ts_tablet_manager.cc:1398] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:38.711937 8529 tablet_bootstrap.cc:492] T 30e7bac7fe5f4cd3895ab29e9fc5986b P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:38.712059 8529 ts_tablet_manager.cc:1367] T 30e7bac7fe5f4cd3895ab29e9fc5986b P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.000s
I20240502 20:46:38.712503 8529 raft_consensus.cc:348] T 30e7bac7fe5f4cd3895ab29e9fc5986b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:38.712617 8529 raft_consensus.cc:374] T 30e7bac7fe5f4cd3895ab29e9fc5986b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:38.712663 8529 raft_consensus.cc:729] T 30e7bac7fe5f4cd3895ab29e9fc5986b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:38.712756 8529 consensus_queue.cc:260] T 30e7bac7fe5f4cd3895ab29e9fc5986b P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:38.713016 8529 ts_tablet_manager.cc:1398] T 30e7bac7fe5f4cd3895ab29e9fc5986b P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:38.715144 8476 tablet_bootstrap.cc:492] T 5f7707f47e43472dbaf966db41f0745a P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:38.715231 8476 ts_tablet_manager.cc:1367] T 5f7707f47e43472dbaf966db41f0745a P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.026s user 0.001s sys 0.001s
I20240502 20:46:38.715552 8761 tablet_bootstrap.cc:492] T ee6f512ddcb64018b0bbae67c9b3854b P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:38.715634 8476 raft_consensus.cc:348] T 5f7707f47e43472dbaf966db41f0745a P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:38.715716 8476 raft_consensus.cc:374] T 5f7707f47e43472dbaf966db41f0745a P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:38.715744 8476 raft_consensus.cc:729] T 5f7707f47e43472dbaf966db41f0745a P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:38.715843 8476 consensus_queue.cc:260] T 5f7707f47e43472dbaf966db41f0745a P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:38.716109 8476 ts_tablet_manager.cc:1398] T 5f7707f47e43472dbaf966db41f0745a P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:38.716475 8761 tablet_bootstrap.cc:654] T ee6f512ddcb64018b0bbae67c9b3854b P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:38.717967 8710 mvcc.cc:205] Tried to move back new op lower bound from 7023340743426232320 to 7023340742707040256. Current Snapshot: MvccSnapshot[applied={T|T < 7023340743426232320}]
I20240502 20:46:38.718160 29034 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=859757995ff74518b2021e2c01d75272] alter complete (version 1)
I20240502 20:46:38.719169 8750 tablet_bootstrap.cc:492] T 5f7707f47e43472dbaf966db41f0745a P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:38.719571 8750 ts_tablet_manager.cc:1367] T 5f7707f47e43472dbaf966db41f0745a P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.000s
I20240502 20:46:38.720032 8750 raft_consensus.cc:348] T 5f7707f47e43472dbaf966db41f0745a P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:38.720129 8750 raft_consensus.cc:374] T 5f7707f47e43472dbaf966db41f0745a P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:38.720161 8750 raft_consensus.cc:729] T 5f7707f47e43472dbaf966db41f0745a P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:38.720171 6616 consensus_queue.cc:1035] T 058bb6623be2473ba4cf90c2ca0d52a7 P 2a9c1af8cf814030bde56b67ca5c2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:38.720264 8750 consensus_queue.cc:260] T 5f7707f47e43472dbaf966db41f0745a P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:38.720523 8750 ts_tablet_manager.cc:1398] T 5f7707f47e43472dbaf966db41f0745a P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:38.721871 8769 tablet.cc:1722] T 7cb63dbbd86c46c38e958e65af9c2c84 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:38.721999 8264 raft_consensus.cc:482] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:38.722086 8264 raft_consensus.cc:504] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:38.722420 8264 leader_election.cc:288] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:38.722632 8769 tablet.cc:1620] T 7cb63dbbd86c46c38e958e65af9c2c84 P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:38.722862 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee6f512ddcb64018b0bbae67c9b3854b" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:38.723001 31495 raft_consensus.cc:2454] T ee6f512ddcb64018b0bbae67c9b3854b P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:38.723028 29039 catalog_manager.cc:5576] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887) does not have the latest schema for tablet 7cb63dbbd86c46c38e958e65af9c2c84 (table rebalancer_dynamic_table [id=859757995ff74518b2021e2c01d75272]). Expected version 1 got 0
I20240502 20:46:38.723152 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee6f512ddcb64018b0bbae67c9b3854b" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:38.723376 31692 leader_election.cc:302] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:38.723729 8264 raft_consensus.cc:2781] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:38.723783 8264 raft_consensus.cc:482] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:38.723817 8264 raft_consensus.cc:3037] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
W20240502 20:46:38.724198 31693 leader_election.cc:341] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): Illegal state: must be running to vote when last-logged opid is not known
I20240502 20:46:38.724779 8264 raft_consensus.cc:504] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:38.725065 8764 tablet.cc:1722] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:38.725138 8264 leader_election.cc:288] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:38.725328 8764 tablet.cc:1620] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6: MemRowSet was empty: no flush needed.
I20240502 20:46:38.725724 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee6f512ddcb64018b0bbae67c9b3854b" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:38.725818 31495 raft_consensus.cc:3037] T ee6f512ddcb64018b0bbae67c9b3854b P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
W20240502 20:46:38.725893 29249 consensus_peers.cc:485] T fe41756b5dc74f7f9105e002b8bc9c84 P 09258ac095664fa99caf0707262cde13 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:38.726655 31495 raft_consensus.cc:2454] T ee6f512ddcb64018b0bbae67c9b3854b P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:38.726969 31692 leader_election.cc:302] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:38.727370 8264 raft_consensus.cc:2781] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:38.727450 8264 raft_consensus.cc:686] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Becoming Leader. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Running, Role: LEADER
I20240502 20:46:38.727564 8264 consensus_queue.cc:237] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:38.727838 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee6f512ddcb64018b0bbae67c9b3854b" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
W20240502 20:46:38.728463 31693 leader_election.cc:341] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): Illegal state: must be running to vote when last-logged opid is not known
I20240502 20:46:38.731676 29034 catalog_manager.cc:5484] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: term changed from 0 to 1, leader changed from <none> to e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71). New cstate: current_term: 1 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:38.734856 8761 tablet_bootstrap.cc:492] T ee6f512ddcb64018b0bbae67c9b3854b P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:38.734956 8761 ts_tablet_manager.cc:1367] T ee6f512ddcb64018b0bbae67c9b3854b P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20240502 20:46:38.735381 8761 raft_consensus.cc:348] T ee6f512ddcb64018b0bbae67c9b3854b P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:38.735466 8761 raft_consensus.cc:374] T ee6f512ddcb64018b0bbae67c9b3854b P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:38.735487 8761 raft_consensus.cc:729] T ee6f512ddcb64018b0bbae67c9b3854b P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:38.735617 8761 consensus_queue.cc:260] T ee6f512ddcb64018b0bbae67c9b3854b P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:38.735989 8761 ts_tablet_manager.cc:1398] T ee6f512ddcb64018b0bbae67c9b3854b P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20240502 20:46:38.787760 31429 consensus_peers.cc:485] T 5993321dc8f54f14a89fa2ba4a44ccc7 P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:38.825112 29035 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:38.831532 29446 raft_consensus.cc:1261] T 378b1151c07648919504bacdcb20a087 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 3 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20240502 20:46:38.831921 5346 consensus_queue.cc:1035] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 20:46:38.832808 8764 tablet.cc:1722] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:38.833141 8769 tablet.cc:1722] T 7cb63dbbd86c46c38e958e65af9c2c84 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:38.833428 8769 tablet.cc:1620] T 7cb63dbbd86c46c38e958e65af9c2c84 P 2a9c1af8cf814030bde56b67ca5c2fbe: MemRowSet was empty: no flush needed.
I20240502 20:46:38.833485 8764 tablet.cc:1620] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6: MemRowSet was empty: no flush needed.
I20240502 20:46:38.842195 8765 tablet.cc:1722] T 7cb63dbbd86c46c38e958e65af9c2c84 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:38.842514 8765 tablet.cc:1620] T 7cb63dbbd86c46c38e958e65af9c2c84 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
W20240502 20:46:38.852267 31691 consensus_peers.cc:485] T 0f5e277eec15411c8c1fd76189893ee9 P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:38.852756 8680 raft_consensus.cc:482] T 5f7707f47e43472dbaf966db41f0745a P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:38.852854 8680 raft_consensus.cc:504] T 5f7707f47e43472dbaf966db41f0745a P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:38.853233 8680 leader_election.cc:288] T 5f7707f47e43472dbaf966db41f0745a P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:38.854085 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f7707f47e43472dbaf966db41f0745a" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:38.854211 29315 raft_consensus.cc:2454] T 5f7707f47e43472dbaf966db41f0745a P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:38.854584 31692 leader_election.cc:302] T 5f7707f47e43472dbaf966db41f0745a P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:38.854739 8522 raft_consensus.cc:2781] T 5f7707f47e43472dbaf966db41f0745a P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:38.854784 8522 raft_consensus.cc:482] T 5f7707f47e43472dbaf966db41f0745a P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:38.854818 8522 raft_consensus.cc:3037] T 5f7707f47e43472dbaf966db41f0745a P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:38.854416 29033 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=859757995ff74518b2021e2c01d75272] alter complete (version 2)
I20240502 20:46:38.855832 8522 raft_consensus.cc:504] T 5f7707f47e43472dbaf966db41f0745a P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:38.856156 8522 leader_election.cc:288] T 5f7707f47e43472dbaf966db41f0745a P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Requested vote from peers e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:38.857738 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f7707f47e43472dbaf966db41f0745a" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:38.857853 29315 raft_consensus.cc:3037] T 5f7707f47e43472dbaf966db41f0745a P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:38.858670 29315 raft_consensus.cc:2454] T 5f7707f47e43472dbaf966db41f0745a P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:38.859156 31692 leader_election.cc:302] T 5f7707f47e43472dbaf966db41f0745a P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:38.860244 8522 raft_consensus.cc:2781] T 5f7707f47e43472dbaf966db41f0745a P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:38.860266 31887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f7707f47e43472dbaf966db41f0745a" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:38.860327 8522 raft_consensus.cc:686] T 5f7707f47e43472dbaf966db41f0745a P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Becoming Leader. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Running, Role: LEADER
I20240502 20:46:38.860285 31886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f7707f47e43472dbaf966db41f0745a" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:38.860389 31887 raft_consensus.cc:3037] T 5f7707f47e43472dbaf966db41f0745a P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:38.860443 8522 consensus_queue.cc:237] T 5f7707f47e43472dbaf966db41f0745a P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:38.861366 31887 raft_consensus.cc:2454] T 5f7707f47e43472dbaf966db41f0745a P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:38.862804 29038 catalog_manager.cc:5484] T 5f7707f47e43472dbaf966db41f0745a P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: term changed from 0 to 1, leader changed from <none> to e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71). New cstate: current_term: 1 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
W20240502 20:46:38.864230 29511 consensus_peers.cc:485] T 378b1151c07648919504bacdcb20a087 P db17f26a73f3447b92aa9a947e3a7057 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:38.864356 31691 consensus_peers.cc:462] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
W20240502 20:46:38.877233 31822 consensus_peers.cc:485] T 03db15f1ebfb4e4ea271d6bb5c76110d P e6b60803d0764d228c863e2da65e3691 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:38.881052 8814 raft_consensus.cc:482] T 30e7bac7fe5f4cd3895ab29e9fc5986b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:38.881139 8814 raft_consensus.cc:504] T 30e7bac7fe5f4cd3895ab29e9fc5986b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:38.881454 8814 leader_election.cc:288] T 30e7bac7fe5f4cd3895ab29e9fc5986b P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:38.882743 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30e7bac7fe5f4cd3895ab29e9fc5986b" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:38.882848 29315 raft_consensus.cc:2454] T 30e7bac7fe5f4cd3895ab29e9fc5986b P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:38.883152 29381 leader_election.cc:302] T 30e7bac7fe5f4cd3895ab29e9fc5986b P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:38.883292 8814 raft_consensus.cc:2781] T 30e7bac7fe5f4cd3895ab29e9fc5986b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:38.883344 8814 raft_consensus.cc:482] T 30e7bac7fe5f4cd3895ab29e9fc5986b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:38.883378 8814 raft_consensus.cc:3037] T 30e7bac7fe5f4cd3895ab29e9fc5986b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:38.883422 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30e7bac7fe5f4cd3895ab29e9fc5986b" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:38.883528 31495 raft_consensus.cc:2454] T 30e7bac7fe5f4cd3895ab29e9fc5986b P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:38.884442 8814 raft_consensus.cc:504] T 30e7bac7fe5f4cd3895ab29e9fc5986b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:38.884769 8814 leader_election.cc:288] T 30e7bac7fe5f4cd3895ab29e9fc5986b P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:38.885071 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30e7bac7fe5f4cd3895ab29e9fc5986b" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:38.885125 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30e7bac7fe5f4cd3895ab29e9fc5986b" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:38.885169 29315 raft_consensus.cc:3037] T 30e7bac7fe5f4cd3895ab29e9fc5986b P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:38.885247 31495 raft_consensus.cc:3037] T 30e7bac7fe5f4cd3895ab29e9fc5986b P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:38.886080 31495 raft_consensus.cc:2454] T 30e7bac7fe5f4cd3895ab29e9fc5986b P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:38.886117 29315 raft_consensus.cc:2454] T 30e7bac7fe5f4cd3895ab29e9fc5986b P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:38.886379 29381 leader_election.cc:302] T 30e7bac7fe5f4cd3895ab29e9fc5986b P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:38.886672 8814 raft_consensus.cc:2781] T 30e7bac7fe5f4cd3895ab29e9fc5986b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:38.886821 8814 raft_consensus.cc:686] T 30e7bac7fe5f4cd3895ab29e9fc5986b P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:38.886977 8814 consensus_queue.cc:237] T 30e7bac7fe5f4cd3895ab29e9fc5986b P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:38.889110 29038 catalog_manager.cc:5484] T 30e7bac7fe5f4cd3895ab29e9fc5986b P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:38.915841 29424 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 20:46:38.916327 31474 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240502 20:46:38.918540 29557 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 20:46:38.921154 31735 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 20:46:38.925004 29295 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:38.934020 29162 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 20:46:38.960942 29038 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
W20240502 20:46:38.962427 29511 consensus_peers.cc:485] T b15554a988034cc7ab18ac2330f1a356 P db17f26a73f3447b92aa9a947e3a7057 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:38.966025 31867 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:38.966446 8764 tablet.cc:1722] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:38.967101 8769 tablet.cc:1722] T 7cb63dbbd86c46c38e958e65af9c2c84 P 2a9c1af8cf814030bde56b67ca5c2fbe: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:38.972887 8765 tablet.cc:1722] T 7cb63dbbd86c46c38e958e65af9c2c84 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:38.973968 29038 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=859757995ff74518b2021e2c01d75272] alter complete (version 3)
W20240502 20:46:38.982240 29511 consensus_peers.cc:485] T 58d4026c208f45d8870a2eaa1c97efab P db17f26a73f3447b92aa9a947e3a7057 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 20:46:39.026644 29380 consensus_peers.cc:485] T 90bab3d11d424b3abcdc74d5da51cbee P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 20:46:39.026693 29027 catalog_manager.cc:4573] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1e16a4014cf24e84abc7a7d42460dea3: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
W20240502 20:46:39.069268 29511 consensus_peers.cc:485] T ab91f7c07fd14fbb8cd8a70bf8f616ab P db17f26a73f3447b92aa9a947e3a7057 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:39.077948 29033 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:39.078104 29033 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:39.082629 29033 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet 7cb63dbbd86c46c38e958e65af9c2c84
I20240502 20:46:39.083148 31474 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cb63dbbd86c46c38e958e65af9c2c84 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:39 UTC) from {username='slave'} at 127.0.0.1:40276
I20240502 20:46:39.083204 29424 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cb63dbbd86c46c38e958e65af9c2c84 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:39 UTC) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:39.083492 8871 tablet_replica.cc:330] T 7cb63dbbd86c46c38e958e65af9c2c84 P 2a9c1af8cf814030bde56b67ca5c2fbe: stopping tablet replica
I20240502 20:46:39.084048 31867 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cb63dbbd86c46c38e958e65af9c2c84 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:39 UTC) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:39.085075 8872 tablet_replica.cc:330] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:39.085425 8872 raft_consensus.cc:2229] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:39.085464 8871 raft_consensus.cc:2229] T 7cb63dbbd86c46c38e958e65af9c2c84 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:39.085602 8871 raft_consensus.cc:2258] T 7cb63dbbd86c46c38e958e65af9c2c84 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:39.086086 8872 raft_consensus.cc:2258] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:39.086699 8872 ts_tablet_manager.cc:1875] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:39.086859 8871 ts_tablet_manager.cc:1875] T 7cb63dbbd86c46c38e958e65af9c2c84 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:39.089708 8871 ts_tablet_manager.cc:1888] T 7cb63dbbd86c46c38e958e65af9c2c84 P 2a9c1af8cf814030bde56b67ca5c2fbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:39.089871 8873 tablet_replica.cc:330] T 7cb63dbbd86c46c38e958e65af9c2c84 P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:39.089982 8871 log.cc:1196] T 7cb63dbbd86c46c38e958e65af9c2c84 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/wals/7cb63dbbd86c46c38e958e65af9c2c84
I20240502 20:46:39.090709 8873 raft_consensus.cc:2229] T 7cb63dbbd86c46c38e958e65af9c2c84 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:39.091238 8871 ts_tablet_manager.cc:1909] T 7cb63dbbd86c46c38e958e65af9c2c84 P 2a9c1af8cf814030bde56b67ca5c2fbe: Deleting consensus metadata
I20240502 20:46:39.091400 8873 raft_consensus.cc:2258] T 7cb63dbbd86c46c38e958e65af9c2c84 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:39.092105 29028 catalog_manager.cc:4830] TS 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887): tablet 7cb63dbbd86c46c38e958e65af9c2c84 (table rebalancer_dynamic_table_new_name [id=859757995ff74518b2021e2c01d75272]) successfully deleted
I20240502 20:46:39.092942 8873 ts_tablet_manager.cc:1875] T 7cb63dbbd86c46c38e958e65af9c2c84 P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:39.093739 8872 ts_tablet_manager.cc:1888] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:39.094152 8872 log.cc:1196] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/7cb63dbbd86c46c38e958e65af9c2c84
I20240502 20:46:39.094604 8872 ts_tablet_manager.cc:1909] T 7cb63dbbd86c46c38e958e65af9c2c84 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting consensus metadata
I20240502 20:46:39.095300 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet 7cb63dbbd86c46c38e958e65af9c2c84 (table rebalancer_dynamic_table_new_name [id=859757995ff74518b2021e2c01d75272]) successfully deleted
I20240502 20:46:39.099241 8873 ts_tablet_manager.cc:1888] T 7cb63dbbd86c46c38e958e65af9c2c84 P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:39.099445 8873 log.cc:1196] T 7cb63dbbd86c46c38e958e65af9c2c84 P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/7cb63dbbd86c46c38e958e65af9c2c84
I20240502 20:46:39.099786 8873 ts_tablet_manager.cc:1909] T 7cb63dbbd86c46c38e958e65af9c2c84 P e6b60803d0764d228c863e2da65e3691: Deleting consensus metadata
I20240502 20:46:39.100533 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet 7cb63dbbd86c46c38e958e65af9c2c84 (table rebalancer_dynamic_table_new_name [id=859757995ff74518b2021e2c01d75272]) successfully deleted
I20240502 20:46:39.127607 29295 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:39.136243 29557 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 20:46:39.137300 29162 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 20:46:39.137678 31474 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240502 20:46:39.147110 29424 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
W20240502 20:46:39.150940 31429 consensus_peers.cc:485] T 08feee781db04883b6dcc030bdaa8afe P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:39.159700 31867 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:39.167493 31735 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 20:46:39.186395 29035 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:39.187981 29175 raft_consensus.cc:3037] T ee6f512ddcb64018b0bbae67c9b3854b P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:39.189680 8264 consensus_queue.cc:1035] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:39.190157 7239 raft_consensus.cc:482] T ee6f512ddcb64018b0bbae67c9b3854b P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:39.190229 7239 raft_consensus.cc:504] T ee6f512ddcb64018b0bbae67c9b3854b P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:39.190555 7239 leader_election.cc:288] T ee6f512ddcb64018b0bbae67c9b3854b P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:39.190930 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee6f512ddcb64018b0bbae67c9b3854b" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:39.191040 31495 raft_consensus.cc:2454] T ee6f512ddcb64018b0bbae67c9b3854b P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:39.191156 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee6f512ddcb64018b0bbae67c9b3854b" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:39.191288 29119 leader_election.cc:302] T ee6f512ddcb64018b0bbae67c9b3854b P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, 7796ea3fba294d17a34386873969431d; no voters:
I20240502 20:46:39.191430 7239 raft_consensus.cc:2781] T ee6f512ddcb64018b0bbae67c9b3854b P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 20:46:39.191468 7239 raft_consensus.cc:482] T ee6f512ddcb64018b0bbae67c9b3854b P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:39.191498 7239 raft_consensus.cc:3037] T ee6f512ddcb64018b0bbae67c9b3854b P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:39.192422 7239 raft_consensus.cc:504] T ee6f512ddcb64018b0bbae67c9b3854b P 7796ea3fba294d17a34386873969431d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:39.192730 7239 leader_election.cc:288] T ee6f512ddcb64018b0bbae67c9b3854b P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 2 election: Requested vote from peers e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:39.193432 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee6f512ddcb64018b0bbae67c9b3854b" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:39.193537 31495 raft_consensus.cc:3037] T ee6f512ddcb64018b0bbae67c9b3854b P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:39.194190 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee6f512ddcb64018b0bbae67c9b3854b" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:39.196946 29175 raft_consensus.cc:1226] T ee6f512ddcb64018b0bbae67c9b3854b P 7796ea3fba294d17a34386873969431d [term 2 FOLLOWER]: Rejecting Update request from peer e7ab6b9053664ee5aa71c7978e26fe6a for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20240502 20:46:39.197301 29557 tablet_service.cc:1467] Processing CreateTablet for tablet eb692a2a562f4507bd21f47a0e80119e (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a5cc695de9744dc19a4ad496eff1d5ad]), partition=
I20240502 20:46:39.197463 31867 tablet_service.cc:1467] Processing CreateTablet for tablet eb692a2a562f4507bd21f47a0e80119e (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a5cc695de9744dc19a4ad496eff1d5ad]), partition=
I20240502 20:46:39.197516 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb692a2a562f4507bd21f47a0e80119e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:39.197687 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb692a2a562f4507bd21f47a0e80119e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:39.199683 29162 tablet_service.cc:1467] Processing CreateTablet for tablet eb692a2a562f4507bd21f47a0e80119e (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a5cc695de9744dc19a4ad496eff1d5ad]), partition=
I20240502 20:46:39.199817 8476 tablet_bootstrap.cc:492] T eb692a2a562f4507bd21f47a0e80119e P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:39.199862 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb692a2a562f4507bd21f47a0e80119e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:39.200799 8476 tablet_bootstrap.cc:654] T eb692a2a562f4507bd21f47a0e80119e P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:39.202579 8761 tablet_bootstrap.cc:492] T eb692a2a562f4507bd21f47a0e80119e P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:39.203405 8761 tablet_bootstrap.cc:654] T eb692a2a562f4507bd21f47a0e80119e P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:39.208911 8476 tablet_bootstrap.cc:492] T eb692a2a562f4507bd21f47a0e80119e P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:39.209003 8476 ts_tablet_manager.cc:1367] T eb692a2a562f4507bd21f47a0e80119e P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240502 20:46:39.209470 8476 raft_consensus.cc:348] T eb692a2a562f4507bd21f47a0e80119e P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:39.209569 8476 raft_consensus.cc:374] T eb692a2a562f4507bd21f47a0e80119e P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:39.209599 8476 raft_consensus.cc:729] T eb692a2a562f4507bd21f47a0e80119e P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:39.209713 8476 consensus_queue.cc:260] T eb692a2a562f4507bd21f47a0e80119e P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:39.210000 8476 ts_tablet_manager.cc:1398] T eb692a2a562f4507bd21f47a0e80119e P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:39.215477 8264 consensus_queue.cc:1046] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:39.215651 8264 raft_consensus.cc:3032] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:39.215688 8264 raft_consensus.cc:729] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Running, Role: LEADER
I20240502 20:46:39.215837 8264 consensus_queue.cc:260] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:39.215993 8264 raft_consensus.cc:3037] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:39.194447 31495 raft_consensus.cc:2454] T ee6f512ddcb64018b0bbae67c9b3854b P 2a9c1af8cf814030bde56b67ca5c2fbe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 2.
I20240502 20:46:39.218947 29119 leader_election.cc:302] T ee6f512ddcb64018b0bbae67c9b3854b P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, 7796ea3fba294d17a34386873969431d; no voters: e7ab6b9053664ee5aa71c7978e26fe6a
I20240502 20:46:39.219107 7239 raft_consensus.cc:2781] T ee6f512ddcb64018b0bbae67c9b3854b P 7796ea3fba294d17a34386873969431d [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:39.219192 7239 raft_consensus.cc:686] T ee6f512ddcb64018b0bbae67c9b3854b P 7796ea3fba294d17a34386873969431d [term 2 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:39.219309 7239 consensus_queue.cc:237] T ee6f512ddcb64018b0bbae67c9b3854b P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:39.219974 8885 tablet_bootstrap.cc:492] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:39.221004 8761 tablet_bootstrap.cc:492] T eb692a2a562f4507bd21f47a0e80119e P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:39.221078 8761 ts_tablet_manager.cc:1367] T eb692a2a562f4507bd21f47a0e80119e P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20240502 20:46:39.221454 8761 raft_consensus.cc:348] T eb692a2a562f4507bd21f47a0e80119e P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:39.221534 8761 raft_consensus.cc:374] T eb692a2a562f4507bd21f47a0e80119e P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:39.221557 8761 raft_consensus.cc:729] T eb692a2a562f4507bd21f47a0e80119e P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:39.221669 8761 consensus_queue.cc:260] T eb692a2a562f4507bd21f47a0e80119e P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:39.221899 8761 ts_tablet_manager.cc:1398] T eb692a2a562f4507bd21f47a0e80119e P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:39.222911 8885 tablet_bootstrap.cc:654] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:39.223038 29035 catalog_manager.cc:5484] T ee6f512ddcb64018b0bbae67c9b3854b P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 1 to 2, leader changed from e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 2 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:39.224157 8885 tablet_bootstrap.cc:492] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:39.224242 8885 ts_tablet_manager.cc:1367] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 20:46:39.224668 8885 raft_consensus.cc:348] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:39.224766 8885 raft_consensus.cc:374] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:39.224797 8885 raft_consensus.cc:729] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:39.224901 8885 consensus_queue.cc:260] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:39.225169 8885 ts_tablet_manager.cc:1398] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20240502 20:46:39.244037 29118 consensus_peers.cc:485] T f5bdf4f57d2c4c828b94c7cc2e3c2847 P 7796ea3fba294d17a34386873969431d -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 20:46:39.252696 31429 consensus_peers.cc:485] T 358e4c761223404fbf0f467f38ce9b25 P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:39.254627 31886 consensus_queue.cc:237] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } }
I20240502 20:46:39.255738 29175 raft_consensus.cc:1261] T 21eab607fca143b8a165d777b50238e1 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:39.255872 29575 raft_consensus.cc:1261] T 21eab607fca143b8a165d777b50238e1 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 20:46:39.256153 6850 consensus_queue.cc:1035] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:39.256322 8753 consensus_queue.cc:1035] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 20:46:39.258026 8753 raft_consensus.cc:2932] T 21eab607fca143b8a165d777b50238e1 P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:39.258445 29175 raft_consensus.cc:2932] T 21eab607fca143b8a165d777b50238e1 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:39.258770 29575 raft_consensus.cc:2932] T 21eab607fca143b8a165d777b50238e1 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:39.261811 29036 catalog_manager.cc:5484] T 21eab607fca143b8a165d777b50238e1 P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: config changed from index 2 to 3, VOTER e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71) evicted. New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } }
I20240502 20:46:39.264614 31735 tablet_service.cc:1514] Processing DeleteTablet for tablet 21eab607fca143b8a165d777b50238e1 with delete_type TABLET_DATA_TOMBSTONED (TS e7ab6b9053664ee5aa71c7978e26fe6a not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:33864
I20240502 20:46:39.265738 8915 tablet_replica.cc:330] T 21eab607fca143b8a165d777b50238e1 P e7ab6b9053664ee5aa71c7978e26fe6a: stopping tablet replica
I20240502 20:46:39.265877 8915 raft_consensus.cc:2229] T 21eab607fca143b8a165d777b50238e1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:39.266005 8915 raft_consensus.cc:2258] T 21eab607fca143b8a165d777b50238e1 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:39.266451 8915 ts_tablet_manager.cc:1875] T 21eab607fca143b8a165d777b50238e1 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 20:46:39.268864 8915 ts_tablet_manager.cc:1888] T 21eab607fca143b8a165d777b50238e1 P e7ab6b9053664ee5aa71c7978e26fe6a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 20:46:39.268996 8915 log.cc:1196] T 21eab607fca143b8a165d777b50238e1 P e7ab6b9053664ee5aa71c7978e26fe6a: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/wals/21eab607fca143b8a165d777b50238e1
I20240502 20:46:39.269496 29026 catalog_manager.cc:4830] TS e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787): tablet 21eab607fca143b8a165d777b50238e1 (table rebalancer_extra_table_32 [id=a863244f6b74429ba652c196f42b4ee0]) successfully deleted
I20240502 20:46:39.323798 8522 consensus_queue.cc:1035] T 5f7707f47e43472dbaf966db41f0745a P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 20:46:39.325542 31691 consensus_peers.cc:462] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:39.327819 8680 consensus_queue.cc:1035] T 5f7707f47e43472dbaf966db41f0745a P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:39.364610 8814 consensus_queue.cc:1035] T 30e7bac7fe5f4cd3895ab29e9fc5986b P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:39.368937 8814 consensus_queue.cc:1035] T 30e7bac7fe5f4cd3895ab29e9fc5986b P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:39.378283 5346 raft_consensus.cc:482] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:39.378376 5346 raft_consensus.cc:504] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:39.378762 5346 leader_election.cc:288] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:39.379089 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb692a2a562f4507bd21f47a0e80119e" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:39.379110 31887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb692a2a562f4507bd21f47a0e80119e" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:39.379209 29175 raft_consensus.cc:2454] T eb692a2a562f4507bd21f47a0e80119e P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:39.379230 31887 raft_consensus.cc:2454] T eb692a2a562f4507bd21f47a0e80119e P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 0.
I20240502 20:46:39.379513 29511 leader_election.cc:302] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db17f26a73f3447b92aa9a947e3a7057, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:39.379676 5346 raft_consensus.cc:2781] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:39.379727 5346 raft_consensus.cc:482] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:39.379766 5346 raft_consensus.cc:3037] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:39.380595 5346 raft_consensus.cc:504] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:39.380936 5346 leader_election.cc:288] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:39.381213 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb692a2a562f4507bd21f47a0e80119e" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:39.381312 29175 raft_consensus.cc:3037] T eb692a2a562f4507bd21f47a0e80119e P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:39.381378 31887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb692a2a562f4507bd21f47a0e80119e" candidate_uuid: "db17f26a73f3447b92aa9a947e3a7057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:39.381489 31887 raft_consensus.cc:3037] T eb692a2a562f4507bd21f47a0e80119e P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:39.382223 31887 raft_consensus.cc:2454] T eb692a2a562f4507bd21f47a0e80119e P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:39.382290 29175 raft_consensus.cc:2454] T eb692a2a562f4507bd21f47a0e80119e P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db17f26a73f3447b92aa9a947e3a7057 in term 1.
I20240502 20:46:39.382663 29513 leader_election.cc:302] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:39.382977 5346 raft_consensus.cc:2781] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:39.383061 5346 raft_consensus.cc:686] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Becoming Leader. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Running, Role: LEADER
I20240502 20:46:39.383184 5346 consensus_queue.cc:237] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:39.385440 29035 catalog_manager.cc:5484] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057 reported cstate change: term changed from 0 to 1, leader changed from <none> to db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "db17f26a73f3447b92aa9a947e3a7057" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:39.399322 29035 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_68"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:39.410842 29557 tablet_service.cc:1467] Processing CreateTablet for tablet 8cb81a9268814d4e8470546f99133884 (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=4ec8f77e3ffd48c2bc6d6bab3f43b264]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:39.411126 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cb81a9268814d4e8470546f99133884. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:39.411895 31474 tablet_service.cc:1467] Processing CreateTablet for tablet 8cb81a9268814d4e8470546f99133884 (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=4ec8f77e3ffd48c2bc6d6bab3f43b264]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:39.411901 31867 tablet_service.cc:1467] Processing CreateTablet for tablet e3a580cfa9144590b332c436d261d885 (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=4ec8f77e3ffd48c2bc6d6bab3f43b264]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:39.412163 31474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cb81a9268814d4e8470546f99133884. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:39.412176 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3a580cfa9144590b332c436d261d885. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:39.413406 8885 tablet_bootstrap.cc:492] T 8cb81a9268814d4e8470546f99133884 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:39.414203 8476 tablet_bootstrap.cc:492] T e3a580cfa9144590b332c436d261d885 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:39.414304 8885 tablet_bootstrap.cc:654] T 8cb81a9268814d4e8470546f99133884 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:39.414453 29424 tablet_service.cc:1467] Processing CreateTablet for tablet 8cb81a9268814d4e8470546f99133884 (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=4ec8f77e3ffd48c2bc6d6bab3f43b264]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:39.414727 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cb81a9268814d4e8470546f99133884. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:39.415119 8476 tablet_bootstrap.cc:654] T e3a580cfa9144590b332c436d261d885 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:39.415841 8885 tablet_bootstrap.cc:492] T 8cb81a9268814d4e8470546f99133884 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:39.415840 31735 tablet_service.cc:1467] Processing CreateTablet for tablet 2be581e8f74c426d845c272f61e2f473 (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=4ec8f77e3ffd48c2bc6d6bab3f43b264]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:39.415926 8885 ts_tablet_manager.cc:1367] T 8cb81a9268814d4e8470546f99133884 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:39.416083 31735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2be581e8f74c426d845c272f61e2f473. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:39.417183 29425 tablet_service.cc:1467] Processing CreateTablet for tablet e3a580cfa9144590b332c436d261d885 (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=4ec8f77e3ffd48c2bc6d6bab3f43b264]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:39.417359 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3a580cfa9144590b332c436d261d885. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:39.418325 8885 raft_consensus.cc:348] T 8cb81a9268814d4e8470546f99133884 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:39.418768 8974 tablet_bootstrap.cc:492] T 8cb81a9268814d4e8470546f99133884 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:39.418424 8885 raft_consensus.cc:374] T 8cb81a9268814d4e8470546f99133884 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:39.419224 8885 raft_consensus.cc:729] T 8cb81a9268814d4e8470546f99133884 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:39.419346 8885 consensus_queue.cc:260] T 8cb81a9268814d4e8470546f99133884 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:39.419457 31475 tablet_service.cc:1467] Processing CreateTablet for tablet e3a580cfa9144590b332c436d261d885 (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=4ec8f77e3ffd48c2bc6d6bab3f43b264]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:39.419605 8885 ts_tablet_manager.cc:1398] T 8cb81a9268814d4e8470546f99133884 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:39.419696 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3a580cfa9144590b332c436d261d885. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:39.420518 8971 tablet_bootstrap.cc:492] T 8cb81a9268814d4e8470546f99133884 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:39.420969 8476 tablet_bootstrap.cc:492] T e3a580cfa9144590b332c436d261d885 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:39.421051 8476 ts_tablet_manager.cc:1367] T e3a580cfa9144590b332c436d261d885 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 20:46:39.421512 8476 raft_consensus.cc:348] T e3a580cfa9144590b332c436d261d885 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:39.421607 8476 raft_consensus.cc:374] T e3a580cfa9144590b332c436d261d885 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:39.421636 8476 raft_consensus.cc:729] T e3a580cfa9144590b332c436d261d885 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:39.421736 8476 consensus_queue.cc:260] T e3a580cfa9144590b332c436d261d885 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:39.421808 8971 tablet_bootstrap.cc:654] T 8cb81a9268814d4e8470546f99133884 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:39.422009 8476 ts_tablet_manager.cc:1398] T e3a580cfa9144590b332c436d261d885 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:39.422230 8974 tablet_bootstrap.cc:654] T 8cb81a9268814d4e8470546f99133884 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:39.423676 8975 tablet_bootstrap.cc:492] T 2be581e8f74c426d845c272f61e2f473 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:39.425848 29162 tablet_service.cc:1467] Processing CreateTablet for tablet 2be581e8f74c426d845c272f61e2f473 (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=4ec8f77e3ffd48c2bc6d6bab3f43b264]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:39.426126 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2be581e8f74c426d845c272f61e2f473. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:39.426754 8975 tablet_bootstrap.cc:654] T 2be581e8f74c426d845c272f61e2f473 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:39.428121 8974 tablet_bootstrap.cc:492] T 8cb81a9268814d4e8470546f99133884 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:39.428195 8974 ts_tablet_manager.cc:1367] T 8cb81a9268814d4e8470546f99133884 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 20:46:39.428632 8974 raft_consensus.cc:348] T 8cb81a9268814d4e8470546f99133884 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:39.428725 8974 raft_consensus.cc:374] T 8cb81a9268814d4e8470546f99133884 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:39.428754 8974 raft_consensus.cc:729] T 8cb81a9268814d4e8470546f99133884 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:39.429030 29557 tablet_service.cc:1467] Processing CreateTablet for tablet 2be581e8f74c426d845c272f61e2f473 (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=4ec8f77e3ffd48c2bc6d6bab3f43b264]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:39.429067 8974 consensus_queue.cc:260] T 8cb81a9268814d4e8470546f99133884 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:39.429255 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2be581e8f74c426d845c272f61e2f473. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:39.429351 8974 ts_tablet_manager.cc:1398] T 8cb81a9268814d4e8470546f99133884 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:39.429505 8974 tablet_bootstrap.cc:492] T e3a580cfa9144590b332c436d261d885 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:39.430502 8974 tablet_bootstrap.cc:654] T e3a580cfa9144590b332c436d261d885 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:39.431309 31475 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240502 20:46:39.431324 8761 tablet_bootstrap.cc:492] T 2be581e8f74c426d845c272f61e2f473 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:39.431574 29295 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:39.431951 8971 tablet_bootstrap.cc:492] T 8cb81a9268814d4e8470546f99133884 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:39.432227 8761 tablet_bootstrap.cc:654] T 2be581e8f74c426d845c272f61e2f473 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:39.432235 8971 ts_tablet_manager.cc:1367] T 8cb81a9268814d4e8470546f99133884 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 20:46:39.432693 8971 raft_consensus.cc:348] T 8cb81a9268814d4e8470546f99133884 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:39.432785 8971 raft_consensus.cc:374] T 8cb81a9268814d4e8470546f99133884 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:39.432811 8971 raft_consensus.cc:729] T 8cb81a9268814d4e8470546f99133884 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:39.432914 8971 consensus_queue.cc:260] T 8cb81a9268814d4e8470546f99133884 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:39.433174 8971 ts_tablet_manager.cc:1398] T 8cb81a9268814d4e8470546f99133884 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:39.433315 8971 tablet_bootstrap.cc:492] T e3a580cfa9144590b332c436d261d885 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:39.433436 8885 tablet_bootstrap.cc:492] T 2be581e8f74c426d845c272f61e2f473 P db17f26a73f3447b92aa9a947e3a7057: Bootstrap starting.
I20240502 20:46:39.434280 8885 tablet_bootstrap.cc:654] T 2be581e8f74c426d845c272f61e2f473 P db17f26a73f3447b92aa9a947e3a7057: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:39.434531 8971 tablet_bootstrap.cc:654] T e3a580cfa9144590b332c436d261d885 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:39.437327 8975 tablet_bootstrap.cc:492] T 2be581e8f74c426d845c272f61e2f473 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:39.437470 8975 ts_tablet_manager.cc:1367] T 2be581e8f74c426d845c272f61e2f473 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 20:46:39.437976 8975 raft_consensus.cc:348] T 2be581e8f74c426d845c272f61e2f473 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:39.438139 8975 raft_consensus.cc:374] T 2be581e8f74c426d845c272f61e2f473 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:39.438196 8975 raft_consensus.cc:729] T 2be581e8f74c426d845c272f61e2f473 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:39.438308 8975 consensus_queue.cc:260] T 2be581e8f74c426d845c272f61e2f473 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:39.439251 8885 tablet_bootstrap.cc:492] T 2be581e8f74c426d845c272f61e2f473 P db17f26a73f3447b92aa9a947e3a7057: No bootstrap required, opened a new log
I20240502 20:46:39.439332 8885 ts_tablet_manager.cc:1367] T 2be581e8f74c426d845c272f61e2f473 P db17f26a73f3447b92aa9a947e3a7057: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20240502 20:46:39.439496 8975 ts_tablet_manager.cc:1398] T 2be581e8f74c426d845c272f61e2f473 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:39.439728 8885 raft_consensus.cc:348] T 2be581e8f74c426d845c272f61e2f473 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:39.439812 8885 raft_consensus.cc:374] T 2be581e8f74c426d845c272f61e2f473 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:39.439842 8885 raft_consensus.cc:729] T 2be581e8f74c426d845c272f61e2f473 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db17f26a73f3447b92aa9a947e3a7057, State: Initialized, Role: FOLLOWER
I20240502 20:46:39.439956 8885 consensus_queue.cc:260] T 2be581e8f74c426d845c272f61e2f473 P db17f26a73f3447b92aa9a947e3a7057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:39.440099 8971 tablet_bootstrap.cc:492] T e3a580cfa9144590b332c436d261d885 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:39.440167 8971 ts_tablet_manager.cc:1367] T e3a580cfa9144590b332c436d261d885 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20240502 20:46:39.440232 8885 ts_tablet_manager.cc:1398] T 2be581e8f74c426d845c272f61e2f473 P db17f26a73f3447b92aa9a947e3a7057: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:39.440447 31735 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 20:46:39.440836 8971 raft_consensus.cc:348] T e3a580cfa9144590b332c436d261d885 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:39.440923 8971 raft_consensus.cc:374] T e3a580cfa9144590b332c436d261d885 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:39.440953 8971 raft_consensus.cc:729] T e3a580cfa9144590b332c436d261d885 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:39.441066 8971 consensus_queue.cc:260] T e3a580cfa9144590b332c436d261d885 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:39.441388 8971 ts_tablet_manager.cc:1398] T e3a580cfa9144590b332c436d261d885 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:39.443876 8761 tablet_bootstrap.cc:492] T 2be581e8f74c426d845c272f61e2f473 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:39.443960 8761 ts_tablet_manager.cc:1367] T 2be581e8f74c426d845c272f61e2f473 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 20:46:39.444314 8761 raft_consensus.cc:348] T 2be581e8f74c426d845c272f61e2f473 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:39.444402 8761 raft_consensus.cc:374] T 2be581e8f74c426d845c272f61e2f473 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:39.444430 8761 raft_consensus.cc:729] T 2be581e8f74c426d845c272f61e2f473 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:39.444552 8761 consensus_queue.cc:260] T 2be581e8f74c426d845c272f61e2f473 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:39.444798 8761 ts_tablet_manager.cc:1398] T 2be581e8f74c426d845c272f61e2f473 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:39.448298 8974 tablet_bootstrap.cc:492] T e3a580cfa9144590b332c436d261d885 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:39.448374 8974 ts_tablet_manager.cc:1367] T e3a580cfa9144590b332c436d261d885 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.002s
I20240502 20:46:39.448745 8974 raft_consensus.cc:348] T e3a580cfa9144590b332c436d261d885 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:39.448843 8974 raft_consensus.cc:374] T e3a580cfa9144590b332c436d261d885 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:39.448868 8974 raft_consensus.cc:729] T e3a580cfa9144590b332c436d261d885 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:39.449084 8974 consensus_queue.cc:260] T e3a580cfa9144590b332c436d261d885 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:39.449390 8974 ts_tablet_manager.cc:1398] T e3a580cfa9144590b332c436d261d885 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:39.459348 29162 tablet_service.cc:1429] Tablet server has 22 leaders and 0 scanners
I20240502 20:46:39.460970 29557 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 20:46:39.461470 31867 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:39.469547 29425 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 20:46:39.484208 7979 raft_consensus.cc:482] T 2be581e8f74c426d845c272f61e2f473 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:39.484303 7979 raft_consensus.cc:504] T 2be581e8f74c426d845c272f61e2f473 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:39.484628 7979 leader_election.cc:288] T 2be581e8f74c426d845c272f61e2f473 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:39.485116 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2be581e8f74c426d845c272f61e2f473" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:39.485229 29574 raft_consensus.cc:2454] T 2be581e8f74c426d845c272f61e2f473 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:39.485499 29118 leader_election.cc:302] T 2be581e8f74c426d845c272f61e2f473 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:39.485786 31757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2be581e8f74c426d845c272f61e2f473" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:39.485917 31757 raft_consensus.cc:2454] T 2be581e8f74c426d845c272f61e2f473 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 0.
I20240502 20:46:39.487104 7979 raft_consensus.cc:2781] T 2be581e8f74c426d845c272f61e2f473 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:39.487190 7979 raft_consensus.cc:482] T 2be581e8f74c426d845c272f61e2f473 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:39.487238 7979 raft_consensus.cc:3037] T 2be581e8f74c426d845c272f61e2f473 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:39.488154 7979 raft_consensus.cc:504] T 2be581e8f74c426d845c272f61e2f473 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:39.488502 7979 leader_election.cc:288] T 2be581e8f74c426d845c272f61e2f473 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Requested vote from peers db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:39.488837 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2be581e8f74c426d845c272f61e2f473" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:39.488948 31756 raft_consensus.cc:3037] T 2be581e8f74c426d845c272f61e2f473 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:39.489904 31756 raft_consensus.cc:2454] T 2be581e8f74c426d845c272f61e2f473 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:39.490355 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2be581e8f74c426d845c272f61e2f473" candidate_uuid: "7796ea3fba294d17a34386873969431d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:39.490447 29574 raft_consensus.cc:3037] T 2be581e8f74c426d845c272f61e2f473 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:39.491127 29574 raft_consensus.cc:2454] T 2be581e8f74c426d845c272f61e2f473 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7796ea3fba294d17a34386873969431d in term 1.
I20240502 20:46:39.491365 29118 leader_election.cc:302] T 2be581e8f74c426d845c272f61e2f473 P 7796ea3fba294d17a34386873969431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, db17f26a73f3447b92aa9a947e3a7057; no voters:
I20240502 20:46:39.491658 7979 raft_consensus.cc:2781] T 2be581e8f74c426d845c272f61e2f473 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:39.491724 7979 raft_consensus.cc:686] T 2be581e8f74c426d845c272f61e2f473 P 7796ea3fba294d17a34386873969431d [term 1 LEADER]: Becoming Leader. State: Replica: 7796ea3fba294d17a34386873969431d, State: Running, Role: LEADER
I20240502 20:46:39.491832 7979 consensus_queue.cc:237] T 2be581e8f74c426d845c272f61e2f473 P 7796ea3fba294d17a34386873969431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:39.495297 29040 catalog_manager.cc:5484] T 2be581e8f74c426d845c272f61e2f473 P 7796ea3fba294d17a34386873969431d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7796ea3fba294d17a34386873969431d (127.22.45.65). New cstate: current_term: 1 leader_uuid: "7796ea3fba294d17a34386873969431d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:39.515437 29040 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
W20240502 20:46:39.516320 31429 consensus_peers.cc:485] T 6e80723ee7624d698e9e0a0bd494e1f0 P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:39.519868 31887 raft_consensus.cc:1261] T eb692a2a562f4507bd21f47a0e80119e P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:39.520253 8816 consensus_queue.cc:1035] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:39.520450 29175 raft_consensus.cc:1261] T eb692a2a562f4507bd21f47a0e80119e P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Refusing update from remote peer db17f26a73f3447b92aa9a947e3a7057: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:39.520771 8816 consensus_queue.cc:1035] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:39.523866 29040 catalog_manager.cc:5576] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207) does not have the latest schema for tablet eb692a2a562f4507bd21f47a0e80119e (table rebalancer_dynamic_table [id=a5cc695de9744dc19a4ad496eff1d5ad]). Expected version 1 got 0
I20240502 20:46:39.524312 29035 catalog_manager.cc:5576] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267) does not have the latest schema for tablet eb692a2a562f4507bd21f47a0e80119e (table rebalancer_dynamic_table [id=a5cc695de9744dc19a4ad496eff1d5ad]). Expected version 1 got 0
I20240502 20:46:39.526126 8984 mvcc.cc:205] Tried to move back new op lower bound from 7023340746829283328 to 7023340746274963456. Current Snapshot: MvccSnapshot[applied={T|T < 7023340746829283328}]
I20240502 20:46:39.526598 8988 tablet.cc:1722] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:39.527364 8988 tablet.cc:1620] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057: MemRowSet was empty: no flush needed.
I20240502 20:46:39.529176 29035 catalog_manager.cc:5576] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267) does not have the latest schema for tablet eb692a2a562f4507bd21f47a0e80119e (table rebalancer_dynamic_table [id=a5cc695de9744dc19a4ad496eff1d5ad]). Expected version 1 got 0
I20240502 20:46:39.530565 8995 tablet.cc:1722] T eb692a2a562f4507bd21f47a0e80119e P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:39.530906 8995 tablet.cc:1620] T eb692a2a562f4507bd21f47a0e80119e P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:39.531487 29035 catalog_manager.cc:5576] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207) does not have the latest schema for tablet eb692a2a562f4507bd21f47a0e80119e (table rebalancer_dynamic_table [id=a5cc695de9744dc19a4ad496eff1d5ad]). Expected version 1 got 0
I20240502 20:46:39.532651 8989 tablet.cc:1722] T eb692a2a562f4507bd21f47a0e80119e P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:39.533037 8989 tablet.cc:1620] T eb692a2a562f4507bd21f47a0e80119e P 7796ea3fba294d17a34386873969431d: MemRowSet was empty: no flush needed.
I20240502 20:46:39.533735 29040 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=a5cc695de9744dc19a4ad496eff1d5ad] alter complete (version 1)
W20240502 20:46:39.548933 31691 consensus_peers.cc:485] T 8d5cca35ba4b47748235d1463ec161d6 P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 20:46:39.562659 31691 consensus_peers.cc:485] T 8705260a889841bfa515683432dfe99d P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Error code: TABLET_NOT_FOUND (6). Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 20:46:39.595999 7116 raft_consensus.cc:482] T 8cb81a9268814d4e8470546f99133884 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:39.596098 7116 raft_consensus.cc:504] T 8cb81a9268814d4e8470546f99133884 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:39.596486 7116 leader_election.cc:288] T 8cb81a9268814d4e8470546f99133884 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:39.596985 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cb81a9268814d4e8470546f99133884" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:39.597133 31495 raft_consensus.cc:2454] T 8cb81a9268814d4e8470546f99133884 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:39.597609 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cb81a9268814d4e8470546f99133884" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057" is_pre_election: true
I20240502 20:46:39.597733 29574 raft_consensus.cc:2454] T 8cb81a9268814d4e8470546f99133884 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:39.598213 29381 leader_election.cc:302] T 8cb81a9268814d4e8470546f99133884 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:39.598407 7116 raft_consensus.cc:2781] T 8cb81a9268814d4e8470546f99133884 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:39.598482 7116 raft_consensus.cc:482] T 8cb81a9268814d4e8470546f99133884 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:39.598528 7116 raft_consensus.cc:3037] T 8cb81a9268814d4e8470546f99133884 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:39.599584 7116 raft_consensus.cc:504] T 8cb81a9268814d4e8470546f99133884 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:39.599900 7116 leader_election.cc:288] T 8cb81a9268814d4e8470546f99133884 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897)
I20240502 20:46:39.600847 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cb81a9268814d4e8470546f99133884" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:39.600984 31495 raft_consensus.cc:3037] T 8cb81a9268814d4e8470546f99133884 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:39.601981 31495 raft_consensus.cc:2454] T 8cb81a9268814d4e8470546f99133884 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:39.602326 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cb81a9268814d4e8470546f99133884" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db17f26a73f3447b92aa9a947e3a7057"
I20240502 20:46:39.602442 29574 raft_consensus.cc:3037] T 8cb81a9268814d4e8470546f99133884 P db17f26a73f3447b92aa9a947e3a7057 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:39.603473 29574 raft_consensus.cc:2454] T 8cb81a9268814d4e8470546f99133884 P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:39.604770 29381 leader_election.cc:302] T 8cb81a9268814d4e8470546f99133884 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:39.604982 8814 raft_consensus.cc:2781] T 8cb81a9268814d4e8470546f99133884 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:39.605065 8814 raft_consensus.cc:686] T 8cb81a9268814d4e8470546f99133884 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:39.605185 8814 consensus_queue.cc:237] T 8cb81a9268814d4e8470546f99133884 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } }
I20240502 20:46:39.609059 29035 catalog_manager.cc:5484] T 8cb81a9268814d4e8470546f99133884 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } }
W20240502 20:46:39.610636 29380 consensus_peers.cc:485] T 40305eb373a542f4a6aba1a55947442b P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:39.638856 29042 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:39.646163 8988 tablet.cc:1722] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:39.646494 8988 tablet.cc:1620] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057: MemRowSet was empty: no flush needed.
I20240502 20:46:39.646776 8995 tablet.cc:1722] T eb692a2a562f4507bd21f47a0e80119e P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:39.647051 8995 tablet.cc:1620] T eb692a2a562f4507bd21f47a0e80119e P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:39.656652 8989 tablet.cc:1722] T eb692a2a562f4507bd21f47a0e80119e P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:39.656940 8989 tablet.cc:1620] T eb692a2a562f4507bd21f47a0e80119e P 7796ea3fba294d17a34386873969431d: MemRowSet was empty: no flush needed.
W20240502 20:46:39.668031 29380 consensus_peers.cc:485] T 653cbec396434b4dac0a1928b21d0ebd P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 20:46:39.669296 29040 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=a5cc695de9744dc19a4ad496eff1d5ad] alter complete (version 2)
W20240502 20:46:39.722505 29380 consensus_peers.cc:485] T 60e8272de78a4c94b77408c4d1d5f60e P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Error code: TABLET_NOT_FOUND (6). Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 20:46:39.774125 29042 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
W20240502 20:46:39.778290 29380 consensus_peers.cc:485] T 846f1d6e39e642cfaabf96ad0c52366d P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 20:46:39.779892 8988 tablet.cc:1722] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:39.780308 8989 tablet.cc:1722] T eb692a2a562f4507bd21f47a0e80119e P 7796ea3fba294d17a34386873969431d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:39.780695 8995 tablet.cc:1722] T eb692a2a562f4507bd21f47a0e80119e P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:39.785447 29035 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=a5cc695de9744dc19a4ad496eff1d5ad] alter complete (version 3)
W20240502 20:46:39.796034 29249 consensus_peers.cc:485] T f3c220fbd7df4cefb467ad88e99941b8 P 09258ac095664fa99caf0707262cde13 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 20:46:39.816301 31493 raft_consensus.cc:1261] T ee6f512ddcb64018b0bbae67c9b3854b P 2a9c1af8cf814030bde56b67ca5c2fbe [term 2 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20240502 20:46:39.816646 7979 consensus_queue.cc:1035] T ee6f512ddcb64018b0bbae67c9b3854b P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:39.817894 8814 raft_consensus.cc:482] T e3a580cfa9144590b332c436d261d885 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:39.818001 8814 raft_consensus.cc:504] T e3a580cfa9144590b332c436d261d885 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:39.818322 8814 leader_election.cc:288] T e3a580cfa9144590b332c436d261d885 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:39.818678 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3a580cfa9144590b332c436d261d885" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:39.818805 31495 raft_consensus.cc:2454] T e3a580cfa9144590b332c436d261d885 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 0.
I20240502 20:46:39.819197 29381 leader_election.cc:302] T e3a580cfa9144590b332c436d261d885 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:39.819357 8814 raft_consensus.cc:2781] T e3a580cfa9144590b332c436d261d885 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:39.819409 8814 raft_consensus.cc:482] T e3a580cfa9144590b332c436d261d885 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:39.819448 8814 raft_consensus.cc:3037] T e3a580cfa9144590b332c436d261d885 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:39.820339 8814 raft_consensus.cc:504] T e3a580cfa9144590b332c436d261d885 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:39.820672 8814 leader_election.cc:288] T e3a580cfa9144590b332c436d261d885 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Requested vote from peers 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:39.821115 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3a580cfa9144590b332c436d261d885" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:39.821231 31495 raft_consensus.cc:3037] T e3a580cfa9144590b332c436d261d885 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:39.822000 31495 raft_consensus.cc:2454] T e3a580cfa9144590b332c436d261d885 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:39.822335 29381 leader_election.cc:302] T e3a580cfa9144590b332c436d261d885 P dbcdc6ac635b44b185c3eba63f66e9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9c1af8cf814030bde56b67ca5c2fbe, dbcdc6ac635b44b185c3eba63f66e9d6; no voters:
I20240502 20:46:39.822515 8814 raft_consensus.cc:2781] T e3a580cfa9144590b332c436d261d885 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:39.822593 8814 raft_consensus.cc:686] T e3a580cfa9144590b332c436d261d885 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 LEADER]: Becoming Leader. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Running, Role: LEADER
I20240502 20:46:39.822703 8814 consensus_queue.cc:237] T e3a580cfa9144590b332c436d261d885 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:39.824170 31756 raft_consensus.cc:1261] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a [term 2 FOLLOWER]: Refusing update from remote peer 7796ea3fba294d17a34386873969431d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20240502 20:46:39.824244 31756 pending_rounds.cc:76] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a: Aborting all ops after (but not including) 0
I20240502 20:46:39.824273 31756 pending_rounds.cc:97] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20240502 20:46:39.824318 31756 raft_consensus.cc:2866] T ee6f512ddcb64018b0bbae67c9b3854b P e7ab6b9053664ee5aa71c7978e26fe6a [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 20:46:39.824492 31887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3a580cfa9144590b332c436d261d885" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:39.824604 31887 raft_consensus.cc:3037] T e3a580cfa9144590b332c436d261d885 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:39.824925 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3a580cfa9144590b332c436d261d885" candidate_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:39.825006 29042 catalog_manager.cc:5484] T e3a580cfa9144590b332c436d261d885 P dbcdc6ac635b44b185c3eba63f66e9d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67). New cstate: current_term: 1 leader_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:39.825601 31887 raft_consensus.cc:2454] T e3a580cfa9144590b332c436d261d885 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcdc6ac635b44b185c3eba63f66e9d6 in term 1.
I20240502 20:46:39.825963 7239 consensus_queue.cc:1035] T ee6f512ddcb64018b0bbae67c9b3854b P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 20:46:39.832782 29511 consensus_peers.cc:485] T 9acc12c9f0464e07965c05285fd334f0 P db17f26a73f3447b92aa9a947e3a7057 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Error code: TABLET_NOT_FOUND (6). Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 20:46:39.852588 29295 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:39.854959 29162 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240502 20:46:39.856316 29425 tablet_service.cc:1429] Tablet server has 22 leaders and 0 scanners
I20240502 20:46:39.856563 31475 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240502 20:46:39.866941 29557 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 20:46:39.871964 31735 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 20:46:39.873526 31867 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 20:46:39.886690 29041 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:39.886822 29041 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:39.889112 29041 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet eb692a2a562f4507bd21f47a0e80119e
I20240502 20:46:39.889560 29557 tablet_service.cc:1514] Processing DeleteTablet for tablet eb692a2a562f4507bd21f47a0e80119e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:39 UTC) from {username='slave'} at 127.0.0.1:47730
I20240502 20:46:39.889760 29162 tablet_service.cc:1514] Processing DeleteTablet for tablet eb692a2a562f4507bd21f47a0e80119e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:39 UTC) from {username='slave'} at 127.0.0.1:33046
I20240502 20:46:39.890784 31867 tablet_service.cc:1514] Processing DeleteTablet for tablet eb692a2a562f4507bd21f47a0e80119e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:39 UTC) from {username='slave'} at 127.0.0.1:40510
W20240502 20:46:39.891714 29380 consensus_peers.cc:485] T db9c5ce233da4b1a9ce70ae04aef7658 P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 20:46:39.894210 9045 tablet_replica.cc:330] T eb692a2a562f4507bd21f47a0e80119e P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:39.895555 9045 raft_consensus.cc:2229] T eb692a2a562f4507bd21f47a0e80119e P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:39.895781 9043 tablet_replica.cc:330] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057: stopping tablet replica
I20240502 20:46:39.895982 9043 raft_consensus.cc:2229] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:39.896032 9045 raft_consensus.cc:2258] T eb692a2a562f4507bd21f47a0e80119e P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:39.896215 9043 raft_consensus.cc:2258] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:39.896823 9043 ts_tablet_manager.cc:1875] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:39.896843 9045 ts_tablet_manager.cc:1875] T eb692a2a562f4507bd21f47a0e80119e P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:39.898603 9044 tablet_replica.cc:330] T eb692a2a562f4507bd21f47a0e80119e P 7796ea3fba294d17a34386873969431d: stopping tablet replica
I20240502 20:46:39.898787 9044 raft_consensus.cc:2229] T eb692a2a562f4507bd21f47a0e80119e P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:39.898953 9044 raft_consensus.cc:2258] T eb692a2a562f4507bd21f47a0e80119e P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:39.899533 9044 ts_tablet_manager.cc:1875] T eb692a2a562f4507bd21f47a0e80119e P 7796ea3fba294d17a34386873969431d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:39.901612 9044 ts_tablet_manager.cc:1888] T eb692a2a562f4507bd21f47a0e80119e P 7796ea3fba294d17a34386873969431d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:39.903251 9045 ts_tablet_manager.cc:1888] T eb692a2a562f4507bd21f47a0e80119e P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:39.903810 9045 log.cc:1196] T eb692a2a562f4507bd21f47a0e80119e P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/eb692a2a562f4507bd21f47a0e80119e
I20240502 20:46:39.904158 9045 ts_tablet_manager.cc:1909] T eb692a2a562f4507bd21f47a0e80119e P e6b60803d0764d228c863e2da65e3691: Deleting consensus metadata
I20240502 20:46:39.905257 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet eb692a2a562f4507bd21f47a0e80119e (table rebalancer_dynamic_table_new_name [id=a5cc695de9744dc19a4ad496eff1d5ad]) successfully deleted
I20240502 20:46:39.905586 9043 ts_tablet_manager.cc:1888] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:39.905661 9043 log.cc:1196] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/wals/eb692a2a562f4507bd21f47a0e80119e
I20240502 20:46:39.905967 9043 ts_tablet_manager.cc:1909] T eb692a2a562f4507bd21f47a0e80119e P db17f26a73f3447b92aa9a947e3a7057: Deleting consensus metadata
I20240502 20:46:39.906522 9044 log.cc:1196] T eb692a2a562f4507bd21f47a0e80119e P 7796ea3fba294d17a34386873969431d: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/wals/eb692a2a562f4507bd21f47a0e80119e
I20240502 20:46:39.906981 9044 ts_tablet_manager.cc:1909] T eb692a2a562f4507bd21f47a0e80119e P 7796ea3fba294d17a34386873969431d: Deleting consensus metadata
I20240502 20:46:39.907402 29027 catalog_manager.cc:4830] TS db17f26a73f3447b92aa9a947e3a7057 (127.22.45.68:45897): tablet eb692a2a562f4507bd21f47a0e80119e (table rebalancer_dynamic_table_new_name [id=a5cc695de9744dc19a4ad496eff1d5ad]) successfully deleted
I20240502 20:46:39.908118 29029 catalog_manager.cc:4830] TS 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): tablet eb692a2a562f4507bd21f47a0e80119e (table rebalancer_dynamic_table_new_name [id=a5cc695de9744dc19a4ad496eff1d5ad]) successfully deleted
W20240502 20:46:39.912616 31691 consensus_peers.cc:462] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
W20240502 20:46:39.924866 29511 consensus_peers.cc:485] T fbd2bd8e142741ba864e2f99610b39d6 P db17f26a73f3447b92aa9a947e3a7057 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 20:46:39.927441 31429 consensus_peers.cc:485] T d0538f41ea0e46edae87b8e4dfa995f0 P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 20:46:39.940846 29380 consensus_peers.cc:485] T 2d72ef6e7e8d473baaff8f527c02332b P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 20:46:39.954635 31691 consensus_peers.cc:485] T d6b1c1798aaf4f1fb3d6ef52503e08be P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 20:46:39.963630 31691 consensus_peers.cc:485] T 6436a7d43dab46bca948438f832245ed P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 20:46:39.972030 29118 consensus_peers.cc:485] T 3d89ea92aec54168997241dd78c6abb1 P 7796ea3fba294d17a34386873969431d -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 20:46:39.977846 29511 consensus_peers.cc:485] T db5e418e67a44480b8d4c302cde3b96c P db17f26a73f3447b92aa9a947e3a7057 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
/home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/tools/rebalancer_tool-test.cc:1203: Failure
Value of: s.ok()
Actual: false
Expected: true
Runtime error: /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu: process exited with non-zero status 1
stdout: Per-server replica distribution summary:
Statistic | Value
-----------------------+-----------
Minimum Replica Count | 0
Maximum Replica Count | 46
Average Replica Count | 22.875000
Per-table replica distribution summary:
Replica Skew | Value
--------------+----------
Minimum | 1
Maximum | 3
Average | 2.904762
stderr: W20240502 20:46:11.014940 31941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:11.015298 31941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:11.018563 31941 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240502 20:46:11.512533 31941 rebalancer_tool.cc:215] running whole-cluster rebalancing
W20240502 20:46:12.023175 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 0c689bc5ef924b3eae28a1e2f63671df: could not find any suitable replica to move from server db17f26a73f3447b92aa9a947e3a7057 to server e7ab6b9053664ee5aa71c7978e26fe6a
W20240502 20:46:12.023401 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 0c689bc5ef924b3eae28a1e2f63671df: could not find any suitable replica to move from server 7796ea3fba294d17a34386873969431d to server 2a9c1af8cf814030bde56b67ca5c2fbe
W20240502 20:46:12.023456 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table bec3bd33882044f38d797093171eeb1d: could not find any suitable replica to move from server 2a9c1af8cf814030bde56b67ca5c2fbe to server 09258ac095664fa99caf0707262cde13
W20240502 20:46:12.023609 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table c5c351da1d6f466a88f1fcf68cd0a72d: could not find any suitable replica to move from server 09258ac095664fa99caf0707262cde13 to server e6b60803d0764d228c863e2da65e3691
W20240502 20:46:12.023669 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 85861b676eab48d695a81e1716f5dadc: could not find any suitable replica to move from server 09258ac095664fa99caf0707262cde13 to server 2a9c1af8cf814030bde56b67ca5c2fbe
W20240502 20:46:12.023741 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 08ce6b9eef5e4b2da494180f1ee1852b: could not find any suitable replica to move from server dbcdc6ac635b44b185c3eba63f66e9d6 to server 2a9c1af8cf814030bde56b67ca5c2fbe
W20240502 20:46:12.023835 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table b209cd0423394c4ea3701bdf9e25d73f: could not find any suitable replica to move from server 09258ac095664fa99caf0707262cde13 to server 315915efcf2b413ca7f9d874338e1b2d
W20240502 20:46:12.023869 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 0c689bc5ef924b3eae28a1e2f63671df: could not find any suitable replica to move from server db17f26a73f3447b92aa9a947e3a7057 to server e6b60803d0764d228c863e2da65e3691
W20240502 20:46:12.023912 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 8011864bab7b4506a89a8bbb019d244f: could not find any suitable replica to move from server dbcdc6ac635b44b185c3eba63f66e9d6 to server e7ab6b9053664ee5aa71c7978e26fe6a
W20240502 20:46:12.023947 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table bec3bd33882044f38d797093171eeb1d: could not find any suitable replica to move from server 7796ea3fba294d17a34386873969431d to server e7ab6b9053664ee5aa71c7978e26fe6a
W20240502 20:46:12.023988 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 5fd0afffa41641b9b8de6545456ff39a: could not find any suitable replica to move from server 09258ac095664fa99caf0707262cde13 to server e6b60803d0764d228c863e2da65e3691
W20240502 20:46:12.024027 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table aec26ee01a7a4b05887071d6b1783fb0: could not find any suitable replica to move from server dbcdc6ac635b44b185c3eba63f66e9d6 to server e6b60803d0764d228c863e2da65e3691
W20240502 20:46:12.024155 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table b1432bc7242c45e38bf75a9c91c6646c: could not find any suitable replica to move from server 7796ea3fba294d17a34386873969431d to server e6b60803d0764d228c863e2da65e3691
W20240502 20:46:12.024194 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 85861b676eab48d695a81e1716f5dadc: could not find any suitable replica to move from server db17f26a73f3447b92aa9a947e3a7057 to server e6b60803d0764d228c863e2da65e3691
W20240502 20:46:12.024240 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 79c2e912ffdb4aae9cf78414231a842e: could not find any suitable replica to move from server db17f26a73f3447b92aa9a947e3a7057 to server 2a9c1af8cf814030bde56b67ca5c2fbe
W20240502 20:46:12.024298 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 4b510e9cf32a4794a9193f7ebac46f93: could not find any suitable replica to move from server 09258ac095664fa99caf0707262cde13 to server 2a9c1af8cf814030bde56b67ca5c2fbe
W20240502 20:46:12.024338 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table fd2f36435a614aa98d7f33e9646f33a3: could not find any suitable replica to move from server dbcdc6ac635b44b185c3eba63f66e9d6 to server 2a9c1af8cf814030bde56b67ca5c2fbe
W20240502 20:46:12.024379 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 8dca593309e7400190f93e57aecd5c8e: could not find any suitable replica to move from server 09258ac095664fa99caf0707262cde13 to server e7ab6b9053664ee5aa71c7978e26fe6a
W20240502 20:46:12.024426 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 0c689bc5ef924b3eae28a1e2f63671df: could not find any suitable replica to move from server 7796ea3fba294d17a34386873969431d to server 315915efcf2b413ca7f9d874338e1b2d
W20240502 20:46:12.024510 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 59a4763b0fad4ad084f27fcea88d8624: could not find any suitable replica to move from server dbcdc6ac635b44b185c3eba63f66e9d6 to server e6b60803d0764d228c863e2da65e3691
W20240502 20:46:12.024549 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table cf326efc9cf1428897a47dd98c65f088: could not find any suitable replica to move from server dbcdc6ac635b44b185c3eba63f66e9d6 to server 315915efcf2b413ca7f9d874338e1b2d
W20240502 20:46:12.024590 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table cc807ad977d146dc84be7ee2897657fc: could not find any suitable replica to move from server db17f26a73f3447b92aa9a947e3a7057 to server e7ab6b9053664ee5aa71c7978e26fe6a
W20240502 20:46:12.024631 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table b684c3e68fbe4841bf5ca5674beb20f1: could not find any suitable replica to move from server db17f26a73f3447b92aa9a947e3a7057 to server e6b60803d0764d228c863e2da65e3691
W20240502 20:46:12.024668 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 02149b4452244d3fbab1649adf739573: could not find any suitable replica to move from server 7796ea3fba294d17a34386873969431d to server 315915efcf2b413ca7f9d874338e1b2d
W20240502 20:46:12.024710 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table e2320262ebe84e399b45c01bf9a7d73d: could not find any suitable replica to move from server 09258ac095664fa99caf0707262cde13 to server 2a9c1af8cf814030bde56b67ca5c2fbe
W20240502 20:46:12.024771 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table b1432bc7242c45e38bf75a9c91c6646c: could not find any suitable replica to move from server dbcdc6ac635b44b185c3eba63f66e9d6 to server 2a9c1af8cf814030bde56b67ca5c2fbe
W20240502 20:46:12.024832 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 79c2e912ffdb4aae9cf78414231a842e: could not find any suitable replica to move from server 09258ac095664fa99caf0707262cde13 to server 315915efcf2b413ca7f9d874338e1b2d
W20240502 20:46:12.024873 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 08ce6b9eef5e4b2da494180f1ee1852b: could not find any suitable replica to move from server db17f26a73f3447b92aa9a947e3a7057 to server e6b60803d0764d228c863e2da65e3691
W20240502 20:46:12.024914 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table fd2f36435a614aa98d7f33e9646f33a3: could not find any suitable replica to move from server 09258ac095664fa99caf0707262cde13 to server 2a9c1af8cf814030bde56b67ca5c2fbe
W20240502 20:46:12.024955 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 8dca593309e7400190f93e57aecd5c8e: could not find any suitable replica to move from server dbcdc6ac635b44b185c3eba63f66e9d6 to server 315915efcf2b413ca7f9d874338e1b2d
W20240502 20:46:12.024995 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 5fd0afffa41641b9b8de6545456ff39a: could not find any suitable replica to move from server 7796ea3fba294d17a34386873969431d to server e7ab6b9053664ee5aa71c7978e26fe6a
I20240502 20:46:12.049878 31941 rebalancer_tool.cc:1074] tablet f19fa5a9a0e840c1bc0bc379a1907383: 'db17f26a73f3447b92aa9a947e3a7057' -> 'e7ab6b9053664ee5aa71c7978e26fe6a' move scheduled
I20240502 20:46:12.078902 31941 rebalancer_tool.cc:1074] tablet 4bfb24d76b434cd5b138f868b1f29508: '7796ea3fba294d17a34386873969431d' -> '315915efcf2b413ca7f9d874338e1b2d' move scheduled
I20240502 20:46:12.108127 31941 rebalancer_tool.cc:1074] tablet 4fd651e80e004dc58d7769f227af58a8: 'dbcdc6ac635b44b185c3eba63f66e9d6' -> '2a9c1af8cf814030bde56b67ca5c2fbe' move scheduled
I20240502 20:46:12.130585 31941 rebalancer_tool.cc:1074] tablet 760d15b2acee4322a26f1f2a965f22cc: '09258ac095664fa99caf0707262cde13' -> 'e6b60803d0764d228c863e2da65e3691' move scheduled
I20240502 20:46:12.153878 31941 rebalancer_tool.cc:1074] tablet c18f9efa9c2043f6b2fad95ad9fd8296: 'db17f26a73f3447b92aa9a947e3a7057' -> 'e7ab6b9053664ee5aa71c7978e26fe6a' move scheduled
I20240502 20:46:12.179960 31941 rebalancer_tool.cc:1074] tablet 2c6dfe6a92284c5f96094eb16c757222: '7796ea3fba294d17a34386873969431d' -> '315915efcf2b413ca7f9d874338e1b2d' move scheduled
I20240502 20:46:12.215337 31941 rebalancer_tool.cc:1074] tablet dcbca797b2d14e76b03e379cfe132597: 'dbcdc6ac635b44b185c3eba63f66e9d6' -> '2a9c1af8cf814030bde56b67ca5c2fbe' move scheduled
I20240502 20:46:12.241505 31941 rebalancer_tool.cc:1074] tablet 94bdf135dd3a47d39e7b76dbfd524386: '09258ac095664fa99caf0707262cde13' -> 'e6b60803d0764d228c863e2da65e3691' move scheduled
I20240502 20:46:12.265527 31941 rebalancer_tool.cc:1074] tablet 56928692a7284f33bd34e4396925c570: 'db17f26a73f3447b92aa9a947e3a7057' -> 'e7ab6b9053664ee5aa71c7978e26fe6a' move scheduled
I20240502 20:46:12.290613 31941 rebalancer_tool.cc:1074] tablet 846f1d6e39e642cfaabf96ad0c52366d: '7796ea3fba294d17a34386873969431d' -> '315915efcf2b413ca7f9d874338e1b2d' move scheduled
I20240502 20:46:12.318516 31941 rebalancer_tool.cc:1074] tablet d167293496fb4334ae670a62a2f80fa9: 'dbcdc6ac635b44b185c3eba63f66e9d6' -> '2a9c1af8cf814030bde56b67ca5c2fbe' move scheduled
I20240502 20:46:12.342551 31941 rebalancer_tool.cc:1074] tablet 62f3301937ba4471afcc327be05486cf: '09258ac095664fa99caf0707262cde13' -> 'e6b60803d0764d228c863e2da65e3691' move scheduled
I20240502 20:46:12.373054 31941 rebalancer_tool.cc:1074] tablet a8916d4762844444bae6fa6f930abbfd: 'db17f26a73f3447b92aa9a947e3a7057' -> 'e7ab6b9053664ee5aa71c7978e26fe6a' move scheduled
I20240502 20:46:12.403182 31941 rebalancer_tool.cc:1074] tablet 2d72ef6e7e8d473baaff8f527c02332b: '7796ea3fba294d17a34386873969431d' -> '315915efcf2b413ca7f9d874338e1b2d' move scheduled
I20240502 20:46:12.447149 31941 rebalancer_tool.cc:1074] tablet 616e6b2cacc54145a29a10cc8584bbdf: 'dbcdc6ac635b44b185c3eba63f66e9d6' -> 'e6b60803d0764d228c863e2da65e3691' move scheduled
I20240502 20:46:12.468883 31941 rebalancer_tool.cc:1074] tablet e01c5b97e2d14976812691b78a5143c5: '09258ac095664fa99caf0707262cde13' -> '2a9c1af8cf814030bde56b67ca5c2fbe' move scheduled
I20240502 20:46:12.499485 31941 rebalancer_tool.cc:1074] tablet 9d2cd90615dc4d1894ffe5e80914bdc0: 'db17f26a73f3447b92aa9a947e3a7057' -> 'e7ab6b9053664ee5aa71c7978e26fe6a' move scheduled
I20240502 20:46:12.565161 31941 rebalancer_tool.cc:1074] tablet b0e8b1909df34fce92f94d6af5f5c3b6: '7796ea3fba294d17a34386873969431d' -> '315915efcf2b413ca7f9d874338e1b2d' move scheduled
I20240502 20:46:12.590178 31941 rebalancer_tool.cc:1074] tablet a0e1f5ce5c544045b556cb31e744b169: 'dbcdc6ac635b44b185c3eba63f66e9d6' -> 'e6b60803d0764d228c863e2da65e3691' move scheduled
I20240502 20:46:12.620546 31941 rebalancer_tool.cc:1074] tablet f62df6fab4f248e7aae0a57a9ed58575: '09258ac095664fa99caf0707262cde13' -> '2a9c1af8cf814030bde56b67ca5c2fbe' move scheduled
I20240502 20:46:13.955420 31941 rebalancer_tool.cc:1123] tablet 4bfb24d76b434cd5b138f868b1f29508: '7796ea3fba294d17a34386873969431d' -> '315915efcf2b413ca7f9d874338e1b2d' move is abandoned: Not found: 4bfb24d76b434cd5b138f868b1f29508: tablet not found
I20240502 20:46:14.495782 31941 rebalancer_tool.cc:1123] tablet 2c6dfe6a92284c5f96094eb16c757222: '7796ea3fba294d17a34386873969431d' -> '315915efcf2b413ca7f9d874338e1b2d' move is abandoned: Not found: 2c6dfe6a92284c5f96094eb16c757222: tablet not found
I20240502 20:46:14.917347 31941 rebalancer_tool.cc:1123] tablet 94bdf135dd3a47d39e7b76dbfd524386: '09258ac095664fa99caf0707262cde13' -> 'e6b60803d0764d228c863e2da65e3691' move is abandoned: Not found: 94bdf135dd3a47d39e7b76dbfd524386: tablet not found
I20240502 20:46:15.541203 31941 rebalancer_tool.cc:1123] tablet 56928692a7284f33bd34e4396925c570: 'db17f26a73f3447b92aa9a947e3a7057' -> 'e7ab6b9053664ee5aa71c7978e26fe6a' move is abandoned: Not found: 56928692a7284f33bd34e4396925c570: tablet not found
I20240502 20:46:16.566200 31941 rebalancer_tool.cc:1123] tablet e01c5b97e2d14976812691b78a5143c5: '09258ac095664fa99caf0707262cde13' -> '2a9c1af8cf814030bde56b67ca5c2fbe' move is abandoned: Not found: e01c5b97e2d14976812691b78a5143c5: tablet not found
I20240502 20:46:16.566303 31941 rebalancer_tool.cc:756] re-synchronizing cluster state
W20240502 20:46:16.910334 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 640d5c5611d54c78844cd067d1b6828a: could not find any suitable replica to move from server e7ab6b9053664ee5aa71c7978e26fe6a to server 7796ea3fba294d17a34386873969431d
W20240502 20:46:16.910656 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 02149b4452244d3fbab1649adf739573: could not find any suitable replica to move from server 7796ea3fba294d17a34386873969431d to server e6b60803d0764d228c863e2da65e3691
W20240502 20:46:16.910717 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 640d5c5611d54c78844cd067d1b6828a: could not find any suitable replica to move from server 2a9c1af8cf814030bde56b67ca5c2fbe to server dbcdc6ac635b44b185c3eba63f66e9d6
W20240502 20:46:16.910758 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table fd2f36435a614aa98d7f33e9646f33a3: could not find any suitable replica to move from server dbcdc6ac635b44b185c3eba63f66e9d6 to server 2a9c1af8cf814030bde56b67ca5c2fbe
W20240502 20:46:16.910810 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 8011864bab7b4506a89a8bbb019d244f: could not find any suitable replica to move from server dbcdc6ac635b44b185c3eba63f66e9d6 to server e6b60803d0764d228c863e2da65e3691
W20240502 20:46:16.910841 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table cc807ad977d146dc84be7ee2897657fc: could not find any suitable replica to move from server 09258ac095664fa99caf0707262cde13 to server e7ab6b9053664ee5aa71c7978e26fe6a
W20240502 20:46:16.910871 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table aec26ee01a7a4b05887071d6b1783fb0: could not find any suitable replica to move from server dbcdc6ac635b44b185c3eba63f66e9d6 to server e7ab6b9053664ee5aa71c7978e26fe6a
W20240502 20:46:16.910919 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 08ce6b9eef5e4b2da494180f1ee1852b: could not find any suitable replica to move from server db17f26a73f3447b92aa9a947e3a7057 to server e7ab6b9053664ee5aa71c7978e26fe6a
W20240502 20:46:16.910969 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table cf326efc9cf1428897a47dd98c65f088: could not find any suitable replica to move from server 7796ea3fba294d17a34386873969431d to server e6b60803d0764d228c863e2da65e3691
W20240502 20:46:16.911010 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table b1432bc7242c45e38bf75a9c91c6646c: could not find any suitable replica to move from server db17f26a73f3447b92aa9a947e3a7057 to server 315915efcf2b413ca7f9d874338e1b2d
W20240502 20:46:16.911034 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 640d5c5611d54c78844cd067d1b6828a: could not find any suitable replica to move from server 315915efcf2b413ca7f9d874338e1b2d to server 09258ac095664fa99caf0707262cde13
W20240502 20:46:16.911072 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 4b510e9cf32a4794a9193f7ebac46f93: could not find any suitable replica to move from server 09258ac095664fa99caf0707262cde13 to server 315915efcf2b413ca7f9d874338e1b2d
W20240502 20:46:16.911113 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 5fd0afffa41641b9b8de6545456ff39a: could not find any suitable replica to move from server db17f26a73f3447b92aa9a947e3a7057 to server 2a9c1af8cf814030bde56b67ca5c2fbe
W20240502 20:46:16.911150 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 3ef2607118b441b4b6682450f6395f6c: could not find any suitable replica to move from server 2a9c1af8cf814030bde56b67ca5c2fbe to server e7ab6b9053664ee5aa71c7978e26fe6a
I20240502 20:46:16.939083 31941 rebalancer_tool.cc:1074] tablet 0f57649f122e45f9b8ef82ec975e5359: '315915efcf2b413ca7f9d874338e1b2d' -> '09258ac095664fa99caf0707262cde13' move scheduled
I20240502 20:46:16.964812 31941 rebalancer_tool.cc:1074] tablet 4c4440d27d9e4070a8e425d26d8a0772: '7796ea3fba294d17a34386873969431d' -> 'e6b60803d0764d228c863e2da65e3691' move scheduled
I20240502 20:46:16.989549 31941 rebalancer_tool.cc:1074] tablet 741da32f7c924490a5fa5933708adf30: 'db17f26a73f3447b92aa9a947e3a7057' -> '2a9c1af8cf814030bde56b67ca5c2fbe' move scheduled
I20240502 20:46:17.023648 31941 rebalancer_tool.cc:1074] tablet 11ee3029cd744581ba9c5c213ca8fd2b: 'e7ab6b9053664ee5aa71c7978e26fe6a' -> '09258ac095664fa99caf0707262cde13' move scheduled
I20240502 20:46:17.334683 31941 rebalancer_tool.cc:1123] tablet f62df6fab4f248e7aae0a57a9ed58575: '09258ac095664fa99caf0707262cde13' -> '2a9c1af8cf814030bde56b67ca5c2fbe' move is abandoned: Not found: f62df6fab4f248e7aae0a57a9ed58575: tablet not found
I20240502 20:46:17.976130 31941 rebalancer_tool.cc:1123] tablet b0e8b1909df34fce92f94d6af5f5c3b6: '7796ea3fba294d17a34386873969431d' -> '315915efcf2b413ca7f9d874338e1b2d' move is abandoned: Not found: b0e8b1909df34fce92f94d6af5f5c3b6: tablet not found
I20240502 20:46:17.991112 31941 rebalancer_tool.cc:1136] tablet c18f9efa9c2043f6b2fad95ad9fd8296: 'db17f26a73f3447b92aa9a947e3a7057' -> 'e7ab6b9053664ee5aa71c7978e26fe6a' move completed: OK
I20240502 20:46:17.997730 31941 rebalancer_tool.cc:1136] tablet 4fd651e80e004dc58d7769f227af58a8: 'dbcdc6ac635b44b185c3eba63f66e9d6' -> '2a9c1af8cf814030bde56b67ca5c2fbe' move completed: OK
I20240502 20:46:18.004244 31941 rebalancer_tool.cc:1136] tablet d167293496fb4334ae670a62a2f80fa9: 'dbcdc6ac635b44b185c3eba63f66e9d6' -> '2a9c1af8cf814030bde56b67ca5c2fbe' move completed: OK
I20240502 20:46:18.019014 31941 rebalancer_tool.cc:1136] tablet f19fa5a9a0e840c1bc0bc379a1907383: 'db17f26a73f3447b92aa9a947e3a7057' -> 'e7ab6b9053664ee5aa71c7978e26fe6a' move completed: OK
I20240502 20:46:18.255198 31941 rebalancer_tool.cc:1136] tablet 760d15b2acee4322a26f1f2a965f22cc: '09258ac095664fa99caf0707262cde13' -> 'e6b60803d0764d228c863e2da65e3691' move completed: OK
I20240502 20:46:18.260358 31941 rebalancer_tool.cc:1136] tablet dcbca797b2d14e76b03e379cfe132597: 'dbcdc6ac635b44b185c3eba63f66e9d6' -> '2a9c1af8cf814030bde56b67ca5c2fbe' move completed: OK
I20240502 20:46:18.266170 31941 rebalancer_tool.cc:1136] tablet a8916d4762844444bae6fa6f930abbfd: 'db17f26a73f3447b92aa9a947e3a7057' -> 'e7ab6b9053664ee5aa71c7978e26fe6a' move completed: OK
I20240502 20:46:18.271427 31941 rebalancer_tool.cc:1136] tablet 846f1d6e39e642cfaabf96ad0c52366d: '7796ea3fba294d17a34386873969431d' -> '315915efcf2b413ca7f9d874338e1b2d' move completed: OK
I20240502 20:46:18.272320 31941 rebalancer_tool.cc:1123] tablet 62f3301937ba4471afcc327be05486cf: '09258ac095664fa99caf0707262cde13' -> 'e6b60803d0764d228c863e2da65e3691' move is abandoned: Not found: 62f3301937ba4471afcc327be05486cf: tablet not found
I20240502 20:46:18.277299 31941 rebalancer_tool.cc:1136] tablet 2d72ef6e7e8d473baaff8f527c02332b: '7796ea3fba294d17a34386873969431d' -> '315915efcf2b413ca7f9d874338e1b2d' move completed: OK
I20240502 20:46:18.549082 31941 rebalancer_tool.cc:1136] tablet 616e6b2cacc54145a29a10cc8584bbdf: 'dbcdc6ac635b44b185c3eba63f66e9d6' -> 'e6b60803d0764d228c863e2da65e3691' move completed: OK
I20240502 20:46:18.572582 31941 rebalancer_tool.cc:1074] tablet 689b7368353a430dbffa1cf44f9b5949: 'dbcdc6ac635b44b185c3eba63f66e9d6' -> '2a9c1af8cf814030bde56b67ca5c2fbe' move scheduled
I20240502 20:46:18.597991 31941 rebalancer_tool.cc:1074] tablet 075d3240e24a4532bbb29a068c00c128: 'e7ab6b9053664ee5aa71c7978e26fe6a' -> 'db17f26a73f3447b92aa9a947e3a7057' move scheduled
I20240502 20:46:18.629417 31941 rebalancer_tool.cc:1074] tablet 8c21a6762eaa4ba4ac1c91a42fb37849: '7796ea3fba294d17a34386873969431d' -> 'e6b60803d0764d228c863e2da65e3691' move scheduled
I20240502 20:46:18.660480 31941 rebalancer_tool.cc:1074] tablet fbd2bd8e142741ba864e2f99610b39d6: 'dbcdc6ac635b44b185c3eba63f66e9d6' -> '315915efcf2b413ca7f9d874338e1b2d' move scheduled
I20240502 20:46:18.696259 31941 rebalancer_tool.cc:1074] tablet f14cf6deefec46c7adb598f6d7cfd649: '2a9c1af8cf814030bde56b67ca5c2fbe' -> '7796ea3fba294d17a34386873969431d' move scheduled
I20240502 20:46:18.731490 31941 rebalancer_tool.cc:1074] tablet 03db15f1ebfb4e4ea271d6bb5c76110d: 'e7ab6b9053664ee5aa71c7978e26fe6a' -> 'db17f26a73f3447b92aa9a947e3a7057' move scheduled
I20240502 20:46:18.759711 31941 rebalancer_tool.cc:1074] tablet d4a2c7b197f948a0b236d5e05e2749b9: 'dbcdc6ac635b44b185c3eba63f66e9d6' -> '315915efcf2b413ca7f9d874338e1b2d' move scheduled
I20240502 20:46:18.787943 31941 rebalancer_tool.cc:1074] tablet 6b675480162c4a9799db51ea4b6504a5: '2a9c1af8cf814030bde56b67ca5c2fbe' -> '09258ac095664fa99caf0707262cde13' move scheduled
I20240502 20:46:18.814241 31941 rebalancer_tool.cc:1074] tablet a2bf196bcab440c48aa4ea893997573b: 'db17f26a73f3447b92aa9a947e3a7057' -> 'e6b60803d0764d228c863e2da65e3691' move scheduled
I20240502 20:46:18.849395 31941 rebalancer_tool.cc:1074] tablet 58d4026c208f45d8870a2eaa1c97efab: '7796ea3fba294d17a34386873969431d' -> 'e7ab6b9053664ee5aa71c7978e26fe6a' move scheduled
I20240502 20:46:18.889755 31941 rebalancer_tool.cc:1074] tablet 4a91eafc5ecf45b09f60d712884b7c79: 'dbcdc6ac635b44b185c3eba63f66e9d6' -> '315915efcf2b413ca7f9d874338e1b2d' move scheduled
I20240502 20:46:22.649529 31941 rebalancer_tool.cc:1136] tablet a0e1f5ce5c544045b556cb31e744b169: 'dbcdc6ac635b44b185c3eba63f66e9d6' -> 'e6b60803d0764d228c863e2da65e3691' move completed: OK
I20240502 20:46:23.005141 31941 rebalancer_tool.cc:1136] tablet 9d2cd90615dc4d1894ffe5e80914bdc0: 'db17f26a73f3447b92aa9a947e3a7057' -> 'e7ab6b9053664ee5aa71c7978e26fe6a' move completed: OK
I20240502 20:46:23.011572 31941 rebalancer_tool.cc:1136] tablet 4c4440d27d9e4070a8e425d26d8a0772: '7796ea3fba294d17a34386873969431d' -> 'e6b60803d0764d228c863e2da65e3691' move completed: OK
I20240502 20:46:23.017367 31941 rebalancer_tool.cc:1136] tablet 11ee3029cd744581ba9c5c213ca8fd2b: 'e7ab6b9053664ee5aa71c7978e26fe6a' -> '09258ac095664fa99caf0707262cde13' move completed: OK
I20240502 20:46:23.040231 31941 rebalancer_tool.cc:1074] tablet 90bab3d11d424b3abcdc74d5da51cbee: 'e6b60803d0764d228c863e2da65e3691' -> 'dbcdc6ac635b44b185c3eba63f66e9d6' move scheduled
I20240502 20:46:23.067763 31941 rebalancer_tool.cc:1074] tablet 7fd046a0f0004e5fa6e0825cb626c2ff: '09258ac095664fa99caf0707262cde13' -> 'e7ab6b9053664ee5aa71c7978e26fe6a' move scheduled
I20240502 20:46:24.459182 31941 rebalancer_tool.cc:1136] tablet 58d4026c208f45d8870a2eaa1c97efab: '7796ea3fba294d17a34386873969431d' -> 'e7ab6b9053664ee5aa71c7978e26fe6a' move completed: OK
I20240502 20:46:24.464514 31941 rebalancer_tool.cc:1136] tablet 03db15f1ebfb4e4ea271d6bb5c76110d: 'e7ab6b9053664ee5aa71c7978e26fe6a' -> 'db17f26a73f3447b92aa9a947e3a7057' move completed: OK
I20240502 20:46:24.469417 31941 rebalancer_tool.cc:1136] tablet f14cf6deefec46c7adb598f6d7cfd649: '2a9c1af8cf814030bde56b67ca5c2fbe' -> '7796ea3fba294d17a34386873969431d' move completed: OK
I20240502 20:46:24.474750 31941 rebalancer_tool.cc:1136] tablet fbd2bd8e142741ba864e2f99610b39d6: 'dbcdc6ac635b44b185c3eba63f66e9d6' -> '315915efcf2b413ca7f9d874338e1b2d' move completed: OK
I20240502 20:46:24.479677 31941 rebalancer_tool.cc:1136] tablet 689b7368353a430dbffa1cf44f9b5949: 'dbcdc6ac635b44b185c3eba63f66e9d6' -> '2a9c1af8cf814030bde56b67ca5c2fbe' move completed: OK
I20240502 20:46:24.968338 31941 rebalancer_tool.cc:1136] tablet 741da32f7c924490a5fa5933708adf30: 'db17f26a73f3447b92aa9a947e3a7057' -> '2a9c1af8cf814030bde56b67ca5c2fbe' move completed: OK
I20240502 20:46:24.973230 31941 rebalancer_tool.cc:1136] tablet 0f57649f122e45f9b8ef82ec975e5359: '315915efcf2b413ca7f9d874338e1b2d' -> '09258ac095664fa99caf0707262cde13' move completed: OK
I20240502 20:46:24.977885 31941 rebalancer_tool.cc:1136] tablet a2bf196bcab440c48aa4ea893997573b: 'db17f26a73f3447b92aa9a947e3a7057' -> 'e6b60803d0764d228c863e2da65e3691' move completed: OK
I20240502 20:46:24.983542 31941 rebalancer_tool.cc:1136] tablet 075d3240e24a4532bbb29a068c00c128: 'e7ab6b9053664ee5aa71c7978e26fe6a' -> 'db17f26a73f3447b92aa9a947e3a7057' move completed: OK
I20240502 20:46:25.012531 31941 rebalancer_tool.cc:1074] tablet 3c3bdc0b9e934ae2b81317a371817563: 'db17f26a73f3447b92aa9a947e3a7057' -> '315915efcf2b413ca7f9d874338e1b2d' move scheduled
I20240502 20:46:25.040089 31941 rebalancer_tool.cc:1074] tablet 63a41aabb95d4e8a9071c7673b257c47: 'e7ab6b9053664ee5aa71c7978e26fe6a' -> 'dbcdc6ac635b44b185c3eba63f66e9d6' move scheduled
I20240502 20:46:25.063771 31941 rebalancer_tool.cc:1074] tablet 28d6a4d9c92141f794d6ee04335cf9b8: '2a9c1af8cf814030bde56b67ca5c2fbe' -> 'e6b60803d0764d228c863e2da65e3691' move scheduled
I20240502 20:46:25.087792 31941 rebalancer_tool.cc:1074] tablet 9acc12c9f0464e07965c05285fd334f0: '09258ac095664fa99caf0707262cde13' -> 'e7ab6b9053664ee5aa71c7978e26fe6a' move scheduled
I20240502 20:46:25.116075 31941 rebalancer_tool.cc:1074] tablet 21ee3d7c6f764d0483b335da40e61efc: 'dbcdc6ac635b44b185c3eba63f66e9d6' -> '2a9c1af8cf814030bde56b67ca5c2fbe' move scheduled
I20240502 20:46:26.921828 31941 rebalancer_tool.cc:1136] tablet 6b675480162c4a9799db51ea4b6504a5: '2a9c1af8cf814030bde56b67ca5c2fbe' -> '09258ac095664fa99caf0707262cde13' move completed: OK
I20240502 20:46:26.926935 31941 rebalancer_tool.cc:1136] tablet d4a2c7b197f948a0b236d5e05e2749b9: 'dbcdc6ac635b44b185c3eba63f66e9d6' -> '315915efcf2b413ca7f9d874338e1b2d' move completed: OK
I20240502 20:46:26.931762 31941 rebalancer_tool.cc:1136] tablet 7fd046a0f0004e5fa6e0825cb626c2ff: '09258ac095664fa99caf0707262cde13' -> 'e7ab6b9053664ee5aa71c7978e26fe6a' move completed: OK
I20240502 20:46:26.936715 31941 rebalancer_tool.cc:1136] tablet 8c21a6762eaa4ba4ac1c91a42fb37849: '7796ea3fba294d17a34386873969431d' -> 'e6b60803d0764d228c863e2da65e3691' move completed: OK
I20240502 20:46:26.963465 31941 rebalancer_tool.cc:1074] tablet 035762cba9014632ad520f4777544641: '09258ac095664fa99caf0707262cde13' -> 'e7ab6b9053664ee5aa71c7978e26fe6a' move scheduled
I20240502 20:46:27.004907 31941 rebalancer_tool.cc:1074] tablet 849c31557dec4043ad36dbddc6811b3f: 'dbcdc6ac635b44b185c3eba63f66e9d6' -> '2a9c1af8cf814030bde56b67ca5c2fbe' move scheduled
I20240502 20:46:27.037967 31941 rebalancer_tool.cc:1074] tablet b28fe6bccf4f48a6a5456e6ead5cc920: '09258ac095664fa99caf0707262cde13' -> '315915efcf2b413ca7f9d874338e1b2d' move scheduled
I20240502 20:46:28.959350 31941 rebalancer_tool.cc:1136] tablet 63a41aabb95d4e8a9071c7673b257c47: 'e7ab6b9053664ee5aa71c7978e26fe6a' -> 'dbcdc6ac635b44b185c3eba63f66e9d6' move completed: OK
I20240502 20:46:28.964488 31941 rebalancer_tool.cc:1136] tablet 3c3bdc0b9e934ae2b81317a371817563: 'db17f26a73f3447b92aa9a947e3a7057' -> '315915efcf2b413ca7f9d874338e1b2d' move completed: OK
I20240502 20:46:28.969775 31941 rebalancer_tool.cc:1136] tablet 90bab3d11d424b3abcdc74d5da51cbee: 'e6b60803d0764d228c863e2da65e3691' -> 'dbcdc6ac635b44b185c3eba63f66e9d6' move completed: OK
I20240502 20:46:28.975333 31941 rebalancer_tool.cc:1136] tablet 4a91eafc5ecf45b09f60d712884b7c79: 'dbcdc6ac635b44b185c3eba63f66e9d6' -> '315915efcf2b413ca7f9d874338e1b2d' move completed: OK
I20240502 20:46:29.001361 31941 rebalancer_tool.cc:1074] tablet 0922994ff62d4e648d5132cb08b6a3c8: 'e6b60803d0764d228c863e2da65e3691' -> 'dbcdc6ac635b44b185c3eba63f66e9d6' move scheduled
I20240502 20:46:29.044550 31941 rebalancer_tool.cc:1074] tablet 3943d49acd0341c1a923d900281f1591: 'dbcdc6ac635b44b185c3eba63f66e9d6' -> 'e6b60803d0764d228c863e2da65e3691' move scheduled
I20240502 20:46:29.077661 31941 rebalancer_tool.cc:1074] tablet e7642b18bcbb467187fc8c8b924b79df: 'dbcdc6ac635b44b185c3eba63f66e9d6' -> '2a9c1af8cf814030bde56b67ca5c2fbe' move scheduled
E20240502 20:46:29.135200 5717 ksck_remote.cc:630] unable to open table rebalancer_dynamic_table: Not found: the table does not exist: table_name: "rebalancer_dynamic_table"
E20240502 20:46:30.777269 6191 ksck_remote.cc:630] unable to open table rebalancer_dynamic_table: Not found: the table does not exist: table_name: "rebalancer_dynamic_table"
I20240502 20:46:30.793171 31941 rebalancer_tool.cc:1136] tablet 9acc12c9f0464e07965c05285fd334f0: '09258ac095664fa99caf0707262cde13' -> 'e7ab6b9053664ee5aa71c7978e26fe6a' move completed: OK
I20240502 20:46:30.798142 31941 rebalancer_tool.cc:1136] tablet 28d6a4d9c92141f794d6ee04335cf9b8: '2a9c1af8cf814030bde56b67ca5c2fbe' -> 'e6b60803d0764d228c863e2da65e3691' move completed: OK
I20240502 20:46:30.823935 31941 rebalancer_tool.cc:1074] tablet db9c5ce233da4b1a9ce70ae04aef7658: '09258ac095664fa99caf0707262cde13' -> '315915efcf2b413ca7f9d874338e1b2d' move scheduled
I20240502 20:46:31.658039 31941 rebalancer_tool.cc:1136] tablet b28fe6bccf4f48a6a5456e6ead5cc920: '09258ac095664fa99caf0707262cde13' -> '315915efcf2b413ca7f9d874338e1b2d' move completed: OK
I20240502 20:46:31.663102 31941 rebalancer_tool.cc:1136] tablet 849c31557dec4043ad36dbddc6811b3f: 'dbcdc6ac635b44b185c3eba63f66e9d6' -> '2a9c1af8cf814030bde56b67ca5c2fbe' move completed: OK
I20240502 20:46:31.668957 31941 rebalancer_tool.cc:1136] tablet e7642b18bcbb467187fc8c8b924b79df: 'dbcdc6ac635b44b185c3eba63f66e9d6' -> '2a9c1af8cf814030bde56b67ca5c2fbe' move completed: OK
I20240502 20:46:31.673758 31941 rebalancer_tool.cc:1136] tablet 035762cba9014632ad520f4777544641: '09258ac095664fa99caf0707262cde13' -> 'e7ab6b9053664ee5aa71c7978e26fe6a' move completed: OK
I20240502 20:46:31.976902 31941 rebalancer_tool.cc:1074] tablet 7e0f811a7bd24ab395364e9461144558: '09258ac095664fa99caf0707262cde13' -> 'dbcdc6ac635b44b185c3eba63f66e9d6' move scheduled
I20240502 20:46:32.631090 31941 rebalancer_tool.cc:1136] tablet 3943d49acd0341c1a923d900281f1591: 'dbcdc6ac635b44b185c3eba63f66e9d6' -> 'e6b60803d0764d228c863e2da65e3691' move completed: OK
I20240502 20:46:32.639600 31941 rebalancer_tool.cc:1136] tablet 0922994ff62d4e648d5132cb08b6a3c8: 'e6b60803d0764d228c863e2da65e3691' -> 'dbcdc6ac635b44b185c3eba63f66e9d6' move completed: OK
I20240502 20:46:32.644474 31941 rebalancer_tool.cc:1136] tablet 21ee3d7c6f764d0483b335da40e61efc: 'dbcdc6ac635b44b185c3eba63f66e9d6' -> '2a9c1af8cf814030bde56b67ca5c2fbe' move completed: OK
I20240502 20:46:32.870782 31941 rebalancer_tool.cc:1136] tablet db9c5ce233da4b1a9ce70ae04aef7658: '09258ac095664fa99caf0707262cde13' -> '315915efcf2b413ca7f9d874338e1b2d' move completed: OK
I20240502 20:46:33.534845 31941 rebalancer_tool.cc:1136] tablet 7e0f811a7bd24ab395364e9461144558: '09258ac095664fa99caf0707262cde13' -> 'dbcdc6ac635b44b185c3eba63f66e9d6' move completed: OK
W20240502 20:46:33.827783 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table c292b10269a84c8e83a750d69bb6f95b: could not find any suitable replica to move from server 2a9c1af8cf814030bde56b67ca5c2fbe to server db17f26a73f3447b92aa9a947e3a7057
W20240502 20:46:33.827965 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 3e2bf7b49258487aa136e5dbba387a46: could not find any suitable replica to move from server 315915efcf2b413ca7f9d874338e1b2d to server 7796ea3fba294d17a34386873969431d
W20240502 20:46:33.828155 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 125bd901c1064f21903d5bf6eefae338: could not find any suitable replica to move from server 2a9c1af8cf814030bde56b67ca5c2fbe to server db17f26a73f3447b92aa9a947e3a7057
W20240502 20:46:33.828190 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 15bc33de19bd4776861b116a13509551: could not find any suitable replica to move from server 315915efcf2b413ca7f9d874338e1b2d to server 09258ac095664fa99caf0707262cde13
W20240502 20:46:33.828220 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 288d696f58ef4fb08ffcc029da0478db: could not find any suitable replica to move from server 2a9c1af8cf814030bde56b67ca5c2fbe to server db17f26a73f3447b92aa9a947e3a7057
W20240502 20:46:33.828251 31941 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 73f9b40350d24e0c95e468bd0878b4d5: could not find any suitable replica to move from server 09258ac095664fa99caf0707262cde13 to server e6b60803d0764d228c863e2da65e3691
I20240502 20:46:33.871536 31941 rebalancer_tool.cc:1074] tablet 3eb48ef3d5404e5e9c302b904b5df2cd: '2a9c1af8cf814030bde56b67ca5c2fbe' -> '09258ac095664fa99caf0707262cde13' move scheduled
I20240502 20:46:33.911170 31941 rebalancer_tool.cc:1074] tablet 21eab607fca143b8a165d777b50238e1: 'e7ab6b9053664ee5aa71c7978e26fe6a' -> '7796ea3fba294d17a34386873969431d' move scheduled
I20240502 20:46:33.940955 31941 rebalancer_tool.cc:1074] tablet 60e8272de78a4c94b77408c4d1d5f60e: 'e6b60803d0764d228c863e2da65e3691' -> '315915efcf2b413ca7f9d874338e1b2d' move scheduled
I20240502 20:46:33.987459 31941 rebalancer_tool.cc:1074] tablet 75ace61b6538462b9bd5c0820eeacca9: '2a9c1af8cf814030bde56b67ca5c2fbe' -> 'dbcdc6ac635b44b185c3eba63f66e9d6' move scheduled
I20240502 20:46:34.023741 31941 rebalancer_tool.cc:1074] tablet db9c5ce233da4b1a9ce70ae04aef7658: 'db17f26a73f3447b92aa9a947e3a7057' -> 'e6b60803d0764d228c863e2da65e3691' move scheduled
I20240502 20:46:34.055869 31941 rebalancer_tool.cc:1074] tablet 370bf1bcfb3343c488954ece2488b4c2: '09258ac095664fa99caf0707262cde13' -> 'e7ab6b9053664ee5aa71c7978e26fe6a' move scheduled
I20240502 20:46:34.082667 31941 rebalancer_tool.cc:1074] tablet 8705260a889841bfa515683432dfe99d: '7796ea3fba294d17a34386873969431d' -> '315915efcf2b413ca7f9d874338e1b2d' move scheduled
I20240502 20:46:34.107944 31941 rebalancer_tool.cc:1074] tablet 43fc3dcc2dd54a11becb8f16ec500df1: 'e6b60803d0764d228c863e2da65e3691' -> 'dbcdc6ac635b44b185c3eba63f66e9d6' move scheduled
I20240502 20:46:34.132846 31941 rebalancer_tool.cc:1074] tablet b28fe6bccf4f48a6a5456e6ead5cc920: 'db17f26a73f3447b92aa9a947e3a7057' -> '2a9c1af8cf814030bde56b67ca5c2fbe' move scheduled
I20240502 20:46:34.166244 31941 rebalancer_tool.cc:1074] tablet 2bec922ff95845d1a13c27526f689e15: '315915efcf2b413ca7f9d874338e1b2d' -> '09258ac095664fa99caf0707262cde13' move scheduled
I20240502 20:46:34.201783 31941 rebalancer_tool.cc:1074] tablet 358e4c761223404fbf0f467f38ce9b25: 'e7ab6b9053664ee5aa71c7978e26fe6a' -> '7796ea3fba294d17a34386873969431d' move scheduled
I20240502 20:46:34.237105 31941 rebalancer_tool.cc:1074] tablet 9acc12c9f0464e07965c05285fd334f0: 'dbcdc6ac635b44b185c3eba63f66e9d6' -> '315915efcf2b413ca7f9d874338e1b2d' move scheduled
I20240502 20:46:34.264521 31941 rebalancer_tool.cc:1074] tablet d5ca50fc288f4957a2d6f04c23bd1491: 'e6b60803d0764d228c863e2da65e3691' -> 'db17f26a73f3447b92aa9a947e3a7057' move scheduled
I20240502 20:46:34.293301 31941 rebalancer_tool.cc:1074] tablet b7c80c3b3a5d4505b7dbbac6bab1de59: '2a9c1af8cf814030bde56b67ca5c2fbe' -> '09258ac095664fa99caf0707262cde13' move scheduled
I20240502 20:46:34.315696 31941 rebalancer_tool.cc:1074] tablet 2227f1ae58a044ea955cffa8d8d8a3f2: 'e7ab6b9053664ee5aa71c7978e26fe6a' -> '7796ea3fba294d17a34386873969431d' move scheduled
I20240502 20:46:34.348187 31941 rebalancer_tool.cc:1074] tablet 73c2e0de3fcf455ba1fecf701d23a05f: 'dbcdc6ac635b44b185c3eba63f66e9d6' -> '315915efcf2b413ca7f9d874338e1b2d' move scheduled
I20240502 20:46:34.376765 31941 rebalancer_tool.cc:1074] tablet c8dc196b2d4e48b195524bc79c07b9eb: 'db17f26a73f3447b92aa9a947e3a7057' -> '2a9c1af8cf814030bde56b67ca5c2fbe' move scheduled
I20240502 20:46:34.410727 31941 rebalancer_tool.cc:1074] tablet 7db3d66689ef4fb2869e94448515ad12: 'e6b60803d0764d228c863e2da65e3691' -> '09258ac095664fa99caf0707262cde13' move scheduled
I20240502 20:46:34.446898 31941 rebalancer_tool.cc:1074] tablet 21ee3d7c6f764d0483b335da40e61efc: '7796ea3fba294d17a34386873969431d' -> 'e7ab6b9053664ee5aa71c7978e26fe6a' move scheduled
E20240502 20:46:34.938530 7671 ksck_remote.cc:630] unable to open table rebalancer_dynamic_table: Not found: the table does not exist: table_name: "rebalancer_dynamic_table"
E20240502 20:46:35.093444 7721 ksck_remote.cc:630] unable to open table rebalancer_dynamic_table_new_name: Not found: the table does not exist: table_name: "rebalancer_dynamic_table_new_name"
I20240502 20:46:35.674224 31941 rebalancer_tool.cc:1123] tablet 358e4c761223404fbf0f467f38ce9b25: 'e7ab6b9053664ee5aa71c7978e26fe6a' -> '7796ea3fba294d17a34386873969431d' move is abandoned: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
I20240502 20:46:39.527061 31941 rebalancer_tool.cc:756] re-synchronizing cluster state
Illegal state: tablet server 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): unacceptable health status UNAVAILABLE
I20240502 20:46:39.991858 29041 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:39.997862 31867 tablet_service.cc:1467] Processing CreateTablet for tablet a891a7338cee4190bd2baca7e00a52e5 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=f5de73c60ecd401186da9a19d47b4698]), partition=
I20240502 20:46:39.997875 29425 tablet_service.cc:1467] Processing CreateTablet for tablet a891a7338cee4190bd2baca7e00a52e5 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=f5de73c60ecd401186da9a19d47b4698]), partition=
I20240502 20:46:39.998111 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a891a7338cee4190bd2baca7e00a52e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:39.998111 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a891a7338cee4190bd2baca7e00a52e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:40.000173 29295 tablet_service.cc:1467] Processing CreateTablet for tablet a891a7338cee4190bd2baca7e00a52e5 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=f5de73c60ecd401186da9a19d47b4698]), partition=
I20240502 20:46:40.000399 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a891a7338cee4190bd2baca7e00a52e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:40.001237 9046 tablet_bootstrap.cc:492] T a891a7338cee4190bd2baca7e00a52e5 P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:40.002120 9046 tablet_bootstrap.cc:654] T a891a7338cee4190bd2baca7e00a52e5 P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:40.002950 9047 tablet_bootstrap.cc:492] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:40.003872 9047 tablet_bootstrap.cc:654] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:40.003926 9046 tablet_bootstrap.cc:492] T a891a7338cee4190bd2baca7e00a52e5 P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:40.004053 9046 ts_tablet_manager.cc:1367] T a891a7338cee4190bd2baca7e00a52e5 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 20:46:40.004487 9046 raft_consensus.cc:348] T a891a7338cee4190bd2baca7e00a52e5 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:40.004596 9046 raft_consensus.cc:374] T a891a7338cee4190bd2baca7e00a52e5 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:40.004659 9046 raft_consensus.cc:729] T a891a7338cee4190bd2baca7e00a52e5 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:40.004809 9046 consensus_queue.cc:260] T a891a7338cee4190bd2baca7e00a52e5 P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:40.004871 9047 tablet_bootstrap.cc:492] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:40.004951 9047 ts_tablet_manager.cc:1367] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:40.005250 9046 ts_tablet_manager.cc:1398] T a891a7338cee4190bd2baca7e00a52e5 P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:40.005380 9047 raft_consensus.cc:348] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:40.005484 9047 raft_consensus.cc:374] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:40.005527 9047 raft_consensus.cc:729] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:40.005640 9047 consensus_queue.cc:260] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:40.005894 9047 ts_tablet_manager.cc:1398] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20240502 20:46:40.011951 29249 consensus_peers.cc:485] T d4a2c7b197f948a0b236d5e05e2749b9 P 09258ac095664fa99caf0707262cde13 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 20:46:40.015827 31429 consensus_peers.cc:485] T 211fa6a23bda4c11bb3106080c3ace40 P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 20:46:40.018687 9048 tablet_bootstrap.cc:492] T a891a7338cee4190bd2baca7e00a52e5 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:40.019045 7979 consensus_queue.cc:1035] T 2be581e8f74c426d845c272f61e2f473 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:40.019868 9048 tablet_bootstrap.cc:654] T a891a7338cee4190bd2baca7e00a52e5 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:40.023924 7979 consensus_queue.cc:1035] T 2be581e8f74c426d845c272f61e2f473 P 7796ea3fba294d17a34386873969431d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:40.026888 9048 tablet_bootstrap.cc:492] T a891a7338cee4190bd2baca7e00a52e5 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:40.027026 9048 ts_tablet_manager.cc:1367] T a891a7338cee4190bd2baca7e00a52e5 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 20:46:40.027568 9048 raft_consensus.cc:348] T a891a7338cee4190bd2baca7e00a52e5 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:40.027724 9048 raft_consensus.cc:374] T a891a7338cee4190bd2baca7e00a52e5 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:40.027801 9048 raft_consensus.cc:729] T a891a7338cee4190bd2baca7e00a52e5 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:40.027993 9048 consensus_queue.cc:260] T a891a7338cee4190bd2baca7e00a52e5 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:40.028350 9048 ts_tablet_manager.cc:1398] T a891a7338cee4190bd2baca7e00a52e5 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20240502 20:46:40.042529 29118 consensus_peers.cc:485] T 4a91eafc5ecf45b09f60d712884b7c79 P 7796ea3fba294d17a34386873969431d -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 20:46:40.044947 8814 consensus_queue.cc:1035] T 8cb81a9268814d4e8470546f99133884 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db17f26a73f3447b92aa9a947e3a7057" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:40.047983 8814 consensus_queue.cc:1035] T 8cb81a9268814d4e8470546f99133884 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 20:46:40.055083 29511 consensus_peers.cc:485] T 9da27cc9c2504b7899da253a88dd1f56 P db17f26a73f3447b92aa9a947e3a7057 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 20:46:40.098239 31429 consensus_peers.cc:485] T 2e4a0e5ef89d447b96b998f085e3495c P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 20:46:40.101368 29249 consensus_peers.cc:485] T f0188e48270242949913034b04dbd3b0 P 09258ac095664fa99caf0707262cde13 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 20:46:40.124511 31691 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111) [suppressed 93 similar messages]
W20240502 20:46:40.149874 31429 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111) [suppressed 122 similar messages]
W20240502 20:46:40.150516 31429 consensus_peers.cc:485] T b699c00ff92b41e9ae24e5982ced3ef9 P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 20:46:40.153714 31691 consensus_peers.cc:485] T 1b2fc2ad5cd24a128562714712e89d89 P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 20:46:40.165330 29380 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111) [suppressed 94 similar messages]
W20240502 20:46:40.166661 31822 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111) [suppressed 65 similar messages]
W20240502 20:46:40.167265 31822 consensus_peers.cc:485] T 3eb48ef3d5404e5e9c302b904b5df2cd P e6b60803d0764d228c863e2da65e3691 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 20:46:40.212931 29511 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111) [suppressed 108 similar messages]
W20240502 20:46:40.213836 29249 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111) [suppressed 52 similar messages]
W20240502 20:46:40.214419 29249 consensus_peers.cc:485] T 9992a7ea196645dc82d2430fcb405254 P 09258ac095664fa99caf0707262cde13 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 20:46:40.215445 29380 consensus_peers.cc:485] T d9ebeb5b36674aaf99f35180a61f9cfc P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 20:46:40.231180 31429 consensus_peers.cc:485] T 2a00833b5b404e63b0f720b5c8fb528d P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 20:46:40.235338 31822 consensus_peers.cc:485] T 8feefea4624e49309cef9ecb952bbe96 P e6b60803d0764d228c863e2da65e3691 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 20:46:40.255671 8753 raft_consensus.cc:482] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:40.255774 8753 raft_consensus.cc:504] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:40.256111 8753 leader_election.cc:288] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:40.256419 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a891a7338cee4190bd2baca7e00a52e5" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:40.256474 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a891a7338cee4190bd2baca7e00a52e5" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:40.256565 29446 raft_consensus.cc:2454] T a891a7338cee4190bd2baca7e00a52e5 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:40.256577 29315 raft_consensus.cc:2454] T a891a7338cee4190bd2baca7e00a52e5 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:40.256830 31823 leader_election.cc:302] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dbcdc6ac635b44b185c3eba63f66e9d6, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:40.256979 8753 raft_consensus.cc:2781] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:40.257041 8753 raft_consensus.cc:482] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:40.257071 8753 raft_consensus.cc:3037] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:40.257912 8753 raft_consensus.cc:504] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
I20240502 20:46:40.258227 8753 leader_election.cc:288] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Requested vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945)
I20240502 20:46:40.258549 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a891a7338cee4190bd2baca7e00a52e5" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:40.258644 29315 raft_consensus.cc:3037] T a891a7338cee4190bd2baca7e00a52e5 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:40.258613 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a891a7338cee4190bd2baca7e00a52e5" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:40.258734 29446 raft_consensus.cc:3037] T a891a7338cee4190bd2baca7e00a52e5 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:40.259392 29315 raft_consensus.cc:2454] T a891a7338cee4190bd2baca7e00a52e5 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:40.259675 29446 raft_consensus.cc:2454] T a891a7338cee4190bd2baca7e00a52e5 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:40.259753 31823 leader_election.cc:302] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:40.259907 8753 raft_consensus.cc:2781] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:40.260074 8753 raft_consensus.cc:686] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Becoming Leader. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Running, Role: LEADER
I20240502 20:46:40.260212 8753 consensus_queue.cc:237] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } }
W20240502 20:46:40.260361 29118 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111) [suppressed 46 similar messages]
I20240502 20:46:40.263386 29040 catalog_manager.cc:5484] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6b60803d0764d228c863e2da65e3691 (127.22.45.72). New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } }
I20240502 20:46:40.269527 7116 consensus_queue.cc:1035] T e3a580cfa9144590b332c436d261d885 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 20:46:40.270872 31822 consensus_peers.cc:485] T 63487637b76a4de89ac3f53c707f074b P e6b60803d0764d228c863e2da65e3691 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 20:46:40.272953 8814 consensus_queue.cc:1035] T e3a580cfa9144590b332c436d261d885 P dbcdc6ac635b44b185c3eba63f66e9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 20:46:40.278445 31691 consensus_peers.cc:485] T 46c0a5b0870c4a099c7c17ebc0a5d34e P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 20:46:40.286505 31822 consensus_peers.cc:485] T 11ee3029cd744581ba9c5c213ca8fd2b P e6b60803d0764d228c863e2da65e3691 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 20:46:40.296983 31429 consensus_peers.cc:485] T 4d11a8f31a294ffdb9a1c3c811a38e77 P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 20:46:40.318365 31429 consensus_peers.cc:485] T 4afde5c1bf754a819f84bcc41212617d P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 20:46:40.321812 29511 consensus_peers.cc:485] T 87670bbef3884cc79a7c7e0533661c12 P db17f26a73f3447b92aa9a947e3a7057 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:40.330957 29040 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_extra_table_70"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:40.336614 29162 tablet_service.cc:1467] Processing CreateTablet for tablet 33a5319c9fda4ff78c160556c93520e3 (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=be74474663ad4801a51f8153db7de314]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:40.336925 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33a5319c9fda4ff78c160556c93520e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:40.336933 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 33a5319c9fda4ff78c160556c93520e3 (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=be74474663ad4801a51f8153db7de314]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:40.337203 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33a5319c9fda4ff78c160556c93520e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:40.337284 31867 tablet_service.cc:1467] Processing CreateTablet for tablet ced8eced1cd04a52a292b4e96bbacc2b (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=be74474663ad4801a51f8153db7de314]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:40.337543 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ced8eced1cd04a52a292b4e96bbacc2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:40.339478 31475 tablet_service.cc:1467] Processing CreateTablet for tablet 33a5319c9fda4ff78c160556c93520e3 (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=be74474663ad4801a51f8153db7de314]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:40.339730 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33a5319c9fda4ff78c160556c93520e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:40.339833 31735 tablet_service.cc:1467] Processing CreateTablet for tablet 86863ea03d0f4f52b7c49693fdd946f2 (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=be74474663ad4801a51f8153db7de314]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:40.340097 31735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86863ea03d0f4f52b7c49693fdd946f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:40.341867 29294 tablet_service.cc:1467] Processing CreateTablet for tablet 86863ea03d0f4f52b7c49693fdd946f2 (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=be74474663ad4801a51f8153db7de314]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:40.342062 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86863ea03d0f4f52b7c49693fdd946f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:40.342494 31736 tablet_service.cc:1467] Processing CreateTablet for tablet ced8eced1cd04a52a292b4e96bbacc2b (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=be74474663ad4801a51f8153db7de314]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:40.342813 31736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ced8eced1cd04a52a292b4e96bbacc2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:40.343888 9048 tablet_bootstrap.cc:492] T 33a5319c9fda4ff78c160556c93520e3 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:40.344506 9064 tablet_bootstrap.cc:492] T 86863ea03d0f4f52b7c49693fdd946f2 P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:40.345116 9048 tablet_bootstrap.cc:654] T 33a5319c9fda4ff78c160556c93520e3 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:40.345505 9064 tablet_bootstrap.cc:654] T 86863ea03d0f4f52b7c49693fdd946f2 P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:40.345958 9047 tablet_bootstrap.cc:492] T ced8eced1cd04a52a292b4e96bbacc2b P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:40.347011 9047 tablet_bootstrap.cc:654] T ced8eced1cd04a52a292b4e96bbacc2b P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:40.347272 9064 tablet_bootstrap.cc:492] T 86863ea03d0f4f52b7c49693fdd946f2 P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:40.347332 9064 ts_tablet_manager.cc:1367] T 86863ea03d0f4f52b7c49693fdd946f2 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:40.347746 9064 raft_consensus.cc:348] T 86863ea03d0f4f52b7c49693fdd946f2 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:40.347916 9064 raft_consensus.cc:374] T 86863ea03d0f4f52b7c49693fdd946f2 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:40.348081 9064 raft_consensus.cc:729] T 86863ea03d0f4f52b7c49693fdd946f2 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:40.348362 29162 tablet_service.cc:1467] Processing CreateTablet for tablet 86863ea03d0f4f52b7c49693fdd946f2 (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=be74474663ad4801a51f8153db7de314]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:40.348605 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86863ea03d0f4f52b7c49693fdd946f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:40.348773 9047 tablet_bootstrap.cc:492] T ced8eced1cd04a52a292b4e96bbacc2b P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:40.348835 9047 ts_tablet_manager.cc:1367] T ced8eced1cd04a52a292b4e96bbacc2b P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:40.348934 9064 consensus_queue.cc:260] T 86863ea03d0f4f52b7c49693fdd946f2 P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:40.349251 9047 raft_consensus.cc:348] T ced8eced1cd04a52a292b4e96bbacc2b P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:40.349305 9064 ts_tablet_manager.cc:1398] T 86863ea03d0f4f52b7c49693fdd946f2 P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:40.349339 9047 raft_consensus.cc:374] T ced8eced1cd04a52a292b4e96bbacc2b P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:40.349367 9047 raft_consensus.cc:729] T ced8eced1cd04a52a292b4e96bbacc2b P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:40.349473 9064 tablet_bootstrap.cc:492] T ced8eced1cd04a52a292b4e96bbacc2b P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:40.349493 9047 consensus_queue.cc:260] T ced8eced1cd04a52a292b4e96bbacc2b P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:40.349783 9047 ts_tablet_manager.cc:1398] T ced8eced1cd04a52a292b4e96bbacc2b P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:40.350509 9064 tablet_bootstrap.cc:654] T ced8eced1cd04a52a292b4e96bbacc2b P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:40.351680 9048 tablet_bootstrap.cc:492] T 33a5319c9fda4ff78c160556c93520e3 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:40.351758 9048 ts_tablet_manager.cc:1367] T 33a5319c9fda4ff78c160556c93520e3 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 20:46:40.352166 9048 raft_consensus.cc:348] T 33a5319c9fda4ff78c160556c93520e3 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:40.352263 9048 raft_consensus.cc:374] T 33a5319c9fda4ff78c160556c93520e3 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:40.352290 9048 raft_consensus.cc:729] T 33a5319c9fda4ff78c160556c93520e3 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:40.352403 9048 consensus_queue.cc:260] T 33a5319c9fda4ff78c160556c93520e3 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:40.352684 9048 ts_tablet_manager.cc:1398] T 33a5319c9fda4ff78c160556c93520e3 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:40.352896 9048 tablet_bootstrap.cc:492] T 86863ea03d0f4f52b7c49693fdd946f2 P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:40.353951 29294 tablet_service.cc:1467] Processing CreateTablet for tablet ced8eced1cd04a52a292b4e96bbacc2b (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=be74474663ad4801a51f8153db7de314]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:40.354161 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ced8eced1cd04a52a292b4e96bbacc2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:40.354185 9048 tablet_bootstrap.cc:654] T 86863ea03d0f4f52b7c49693fdd946f2 P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:40.356274 9063 tablet_bootstrap.cc:492] T 33a5319c9fda4ff78c160556c93520e3 P 2a9c1af8cf814030bde56b67ca5c2fbe: Bootstrap starting.
I20240502 20:46:40.357455 9063 tablet_bootstrap.cc:654] T 33a5319c9fda4ff78c160556c93520e3 P 2a9c1af8cf814030bde56b67ca5c2fbe: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:40.357946 9064 tablet_bootstrap.cc:492] T ced8eced1cd04a52a292b4e96bbacc2b P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:40.358026 9064 ts_tablet_manager.cc:1367] T ced8eced1cd04a52a292b4e96bbacc2b P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20240502 20:46:40.358420 9064 raft_consensus.cc:348] T ced8eced1cd04a52a292b4e96bbacc2b P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:40.358636 9062 tablet_bootstrap.cc:492] T 33a5319c9fda4ff78c160556c93520e3 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:40.358654 9064 raft_consensus.cc:374] T ced8eced1cd04a52a292b4e96bbacc2b P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:40.358716 9064 raft_consensus.cc:729] T ced8eced1cd04a52a292b4e96bbacc2b P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:40.358860 9064 consensus_queue.cc:260] T ced8eced1cd04a52a292b4e96bbacc2b P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:40.359194 9064 ts_tablet_manager.cc:1398] T ced8eced1cd04a52a292b4e96bbacc2b P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:40.359470 9062 tablet_bootstrap.cc:654] T 33a5319c9fda4ff78c160556c93520e3 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:40.360704 9048 tablet_bootstrap.cc:492] T 86863ea03d0f4f52b7c49693fdd946f2 P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:40.360848 9063 tablet_bootstrap.cc:492] T 33a5319c9fda4ff78c160556c93520e3 P 2a9c1af8cf814030bde56b67ca5c2fbe: No bootstrap required, opened a new log
I20240502 20:46:40.360837 9048 ts_tablet_manager.cc:1367] T 86863ea03d0f4f52b7c49693fdd946f2 P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20240502 20:46:40.360934 9063 ts_tablet_manager.cc:1367] T 33a5319c9fda4ff78c160556c93520e3 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 20:46:40.361340 9063 raft_consensus.cc:348] T 33a5319c9fda4ff78c160556c93520e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:40.361433 9063 raft_consensus.cc:374] T 33a5319c9fda4ff78c160556c93520e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:40.361455 9063 raft_consensus.cc:729] T 33a5319c9fda4ff78c160556c93520e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9c1af8cf814030bde56b67ca5c2fbe, State: Initialized, Role: FOLLOWER
I20240502 20:46:40.361560 9063 consensus_queue.cc:260] T 33a5319c9fda4ff78c160556c93520e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:40.361793 9063 ts_tablet_manager.cc:1398] T 33a5319c9fda4ff78c160556c93520e3 P 2a9c1af8cf814030bde56b67ca5c2fbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:40.361276 9048 raft_consensus.cc:348] T 86863ea03d0f4f52b7c49693fdd946f2 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:40.362077 9048 raft_consensus.cc:374] T 86863ea03d0f4f52b7c49693fdd946f2 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:40.362151 9048 raft_consensus.cc:729] T 86863ea03d0f4f52b7c49693fdd946f2 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:40.362294 9048 consensus_queue.cc:260] T 86863ea03d0f4f52b7c49693fdd946f2 P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:40.363806 9048 ts_tablet_manager.cc:1398] T 86863ea03d0f4f52b7c49693fdd946f2 P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240502 20:46:40.364058 9048 tablet_bootstrap.cc:492] T ced8eced1cd04a52a292b4e96bbacc2b P 09258ac095664fa99caf0707262cde13: Bootstrap starting.
I20240502 20:46:40.364858 9048 tablet_bootstrap.cc:654] T ced8eced1cd04a52a292b4e96bbacc2b P 09258ac095664fa99caf0707262cde13: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:40.367048 9048 tablet_bootstrap.cc:492] T ced8eced1cd04a52a292b4e96bbacc2b P 09258ac095664fa99caf0707262cde13: No bootstrap required, opened a new log
I20240502 20:46:40.367175 9048 ts_tablet_manager.cc:1367] T ced8eced1cd04a52a292b4e96bbacc2b P 09258ac095664fa99caf0707262cde13: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 20:46:40.367594 9048 raft_consensus.cc:348] T ced8eced1cd04a52a292b4e96bbacc2b P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:40.367749 9048 raft_consensus.cc:374] T ced8eced1cd04a52a292b4e96bbacc2b P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:40.367796 9048 raft_consensus.cc:729] T ced8eced1cd04a52a292b4e96bbacc2b P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09258ac095664fa99caf0707262cde13, State: Initialized, Role: FOLLOWER
I20240502 20:46:40.368216 9048 consensus_queue.cc:260] T ced8eced1cd04a52a292b4e96bbacc2b P 09258ac095664fa99caf0707262cde13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:40.369212 9048 ts_tablet_manager.cc:1398] T ced8eced1cd04a52a292b4e96bbacc2b P 09258ac095664fa99caf0707262cde13: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:40.376581 9062 tablet_bootstrap.cc:492] T 33a5319c9fda4ff78c160556c93520e3 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:40.376690 9062 ts_tablet_manager.cc:1367] T 33a5319c9fda4ff78c160556c93520e3 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240502 20:46:40.377125 9062 raft_consensus.cc:348] T 33a5319c9fda4ff78c160556c93520e3 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:40.377265 9062 raft_consensus.cc:374] T 33a5319c9fda4ff78c160556c93520e3 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:40.377319 9062 raft_consensus.cc:729] T 33a5319c9fda4ff78c160556c93520e3 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:40.377476 9062 consensus_queue.cc:260] T 33a5319c9fda4ff78c160556c93520e3 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:40.377774 9062 ts_tablet_manager.cc:1398] T 33a5319c9fda4ff78c160556c93520e3 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:40.378007 9062 tablet_bootstrap.cc:492] T 86863ea03d0f4f52b7c49693fdd946f2 P 7796ea3fba294d17a34386873969431d: Bootstrap starting.
I20240502 20:46:40.379297 9062 tablet_bootstrap.cc:654] T 86863ea03d0f4f52b7c49693fdd946f2 P 7796ea3fba294d17a34386873969431d: Neither blocks nor log segments found. Creating new log.
W20240502 20:46:40.381327 31429 consensus_peers.cc:485] T b718cecdc692408bbeea91bc9fe21fcb P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 20:46:40.382133 8753 raft_consensus.cc:482] T ced8eced1cd04a52a292b4e96bbacc2b P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:40.382217 8753 raft_consensus.cc:504] T ced8eced1cd04a52a292b4e96bbacc2b P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:40.382985 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ced8eced1cd04a52a292b4e96bbacc2b" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" is_pre_election: true
I20240502 20:46:40.383101 31756 raft_consensus.cc:2454] T ced8eced1cd04a52a292b4e96bbacc2b P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:40.383348 31821 leader_election.cc:302] T ced8eced1cd04a52a292b4e96bbacc2b P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e6b60803d0764d228c863e2da65e3691, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:40.383522 9057 raft_consensus.cc:2781] T ced8eced1cd04a52a292b4e96bbacc2b P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:40.383574 9057 raft_consensus.cc:482] T ced8eced1cd04a52a292b4e96bbacc2b P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:40.383606 9057 raft_consensus.cc:3037] T ced8eced1cd04a52a292b4e96bbacc2b P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:40.384531 8753 leader_election.cc:288] T ced8eced1cd04a52a292b4e96bbacc2b P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:40.384573 9057 raft_consensus.cc:504] T ced8eced1cd04a52a292b4e96bbacc2b P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:40.384877 9057 leader_election.cc:288] T ced8eced1cd04a52a292b4e96bbacc2b P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71:36787)
I20240502 20:46:40.385114 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ced8eced1cd04a52a292b4e96bbacc2b" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:40.385212 29315 raft_consensus.cc:2454] T ced8eced1cd04a52a292b4e96bbacc2b P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 0.
I20240502 20:46:40.385251 29314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ced8eced1cd04a52a292b4e96bbacc2b" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:40.385345 29314 raft_consensus.cc:3037] T ced8eced1cd04a52a292b4e96bbacc2b P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:40.386217 29314 raft_consensus.cc:2454] T ced8eced1cd04a52a292b4e96bbacc2b P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
W20240502 20:46:40.387517 31691 consensus_peers.cc:485] T 5156cc195fcd4b28b9c7bf1aa8a1409e P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 20:46:40.387722 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ced8eced1cd04a52a292b4e96bbacc2b" candidate_uuid: "e6b60803d0764d228c863e2da65e3691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a"
I20240502 20:46:40.387809 31756 raft_consensus.cc:3037] T ced8eced1cd04a52a292b4e96bbacc2b P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:40.388664 31756 raft_consensus.cc:2454] T ced8eced1cd04a52a292b4e96bbacc2b P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6b60803d0764d228c863e2da65e3691 in term 1.
I20240502 20:46:40.389153 31823 leader_election.cc:302] T ced8eced1cd04a52a292b4e96bbacc2b P e6b60803d0764d228c863e2da65e3691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, e6b60803d0764d228c863e2da65e3691; no voters:
I20240502 20:46:40.389812 8753 raft_consensus.cc:2781] T ced8eced1cd04a52a292b4e96bbacc2b P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:40.389900 8753 raft_consensus.cc:686] T ced8eced1cd04a52a292b4e96bbacc2b P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Becoming Leader. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Running, Role: LEADER
I20240502 20:46:40.390019 8753 consensus_queue.cc:237] T ced8eced1cd04a52a292b4e96bbacc2b P e6b60803d0764d228c863e2da65e3691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
W20240502 20:46:40.391940 29027 catalog_manager.cc:4573] TS 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): CreateTablet RPC failed for tablet ab91f7c07fd14fbb8cd8a70bf8f616ab: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
W20240502 20:46:40.392135 29511 consensus_peers.cc:485] T 2bec922ff95845d1a13c27526f689e15 P db17f26a73f3447b92aa9a947e3a7057 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 20:46:40.392900 29041 catalog_manager.cc:5484] T ced8eced1cd04a52a292b4e96bbacc2b P e6b60803d0764d228c863e2da65e3691 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6b60803d0764d228c863e2da65e3691 (127.22.45.72). New cstate: current_term: 1 leader_uuid: "e6b60803d0764d228c863e2da65e3691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
I20240502 20:46:40.393815 9062 tablet_bootstrap.cc:492] T 86863ea03d0f4f52b7c49693fdd946f2 P 7796ea3fba294d17a34386873969431d: No bootstrap required, opened a new log
I20240502 20:46:40.393990 29042 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 20:46:40.397275 29314 raft_consensus.cc:1261] T a891a7338cee4190bd2baca7e00a52e5 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:40.397372 29446 raft_consensus.cc:1261] T a891a7338cee4190bd2baca7e00a52e5 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Refusing update from remote peer e6b60803d0764d228c863e2da65e3691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:40.397735 9057 consensus_queue.cc:1035] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:40.397976 9057 consensus_queue.cc:1035] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:40.400792 29041 catalog_manager.cc:5576] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095) does not have the latest schema for tablet a891a7338cee4190bd2baca7e00a52e5 (table rebalancer_dynamic_table [id=f5de73c60ecd401186da9a19d47b4698]). Expected version 1 got 0
I20240502 20:46:40.402056 29042 catalog_manager.cc:5576] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945) does not have the latest schema for tablet a891a7338cee4190bd2baca7e00a52e5 (table rebalancer_dynamic_table [id=f5de73c60ecd401186da9a19d47b4698]). Expected version 1 got 0
I20240502 20:46:40.402632 9062 ts_tablet_manager.cc:1367] T 86863ea03d0f4f52b7c49693fdd946f2 P 7796ea3fba294d17a34386873969431d: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20240502 20:46:40.403249 9062 raft_consensus.cc:348] T 86863ea03d0f4f52b7c49693fdd946f2 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:40.403401 9062 raft_consensus.cc:374] T 86863ea03d0f4f52b7c49693fdd946f2 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:40.403442 9062 raft_consensus.cc:729] T 86863ea03d0f4f52b7c49693fdd946f2 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7796ea3fba294d17a34386873969431d, State: Initialized, Role: FOLLOWER
I20240502 20:46:40.403563 9062 consensus_queue.cc:260] T 86863ea03d0f4f52b7c49693fdd946f2 P 7796ea3fba294d17a34386873969431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:40.403832 9062 ts_tablet_manager.cc:1398] T 86863ea03d0f4f52b7c49693fdd946f2 P 7796ea3fba294d17a34386873969431d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:40.403937 9080 tablet.cc:1722] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:40.405130 9080 tablet.cc:1620] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:40.405933 9081 tablet.cc:1722] T a891a7338cee4190bd2baca7e00a52e5 P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:40.406213 9081 tablet.cc:1620] T a891a7338cee4190bd2baca7e00a52e5 P dbcdc6ac635b44b185c3eba63f66e9d6: MemRowSet was empty: no flush needed.
I20240502 20:46:40.407939 29040 catalog_manager.cc:5576] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095) does not have the latest schema for tablet a891a7338cee4190bd2baca7e00a52e5 (table rebalancer_dynamic_table [id=f5de73c60ecd401186da9a19d47b4698]). Expected version 1 got 0
I20240502 20:46:40.411878 7910 raft_consensus.cc:482] T 33a5319c9fda4ff78c160556c93520e3 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:40.411957 7910 raft_consensus.cc:504] T 33a5319c9fda4ff78c160556c93520e3 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:40.412294 7910 leader_election.cc:288] T 33a5319c9fda4ff78c160556c93520e3 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:40.412696 29040 catalog_manager.cc:5576] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945) does not have the latest schema for tablet a891a7338cee4190bd2baca7e00a52e5 (table rebalancer_dynamic_table [id=f5de73c60ecd401186da9a19d47b4698]). Expected version 1 got 0
I20240502 20:46:40.412760 31493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33a5319c9fda4ff78c160556c93520e3" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" is_pre_election: true
I20240502 20:46:40.412868 31493 raft_consensus.cc:2454] T 33a5319c9fda4ff78c160556c93520e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:40.413234 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33a5319c9fda4ff78c160556c93520e3" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:40.413339 29182 raft_consensus.cc:2454] T 33a5319c9fda4ff78c160556c93520e3 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 0.
I20240502 20:46:40.413426 29250 leader_election.cc:302] T 33a5319c9fda4ff78c160556c93520e3 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 2a9c1af8cf814030bde56b67ca5c2fbe; no voters:
I20240502 20:46:40.413808 7910 raft_consensus.cc:2781] T 33a5319c9fda4ff78c160556c93520e3 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:40.413861 7910 raft_consensus.cc:482] T 33a5319c9fda4ff78c160556c93520e3 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:40.413894 7910 raft_consensus.cc:3037] T 33a5319c9fda4ff78c160556c93520e3 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
W20240502 20:46:40.414413 31822 consensus_peers.cc:485] T 7a8b04cb606f423f8dd1d489d47408d2 P e6b60803d0764d228c863e2da65e3691 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 20:46:40.414824 7910 raft_consensus.cc:504] T 33a5319c9fda4ff78c160556c93520e3 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
I20240502 20:46:40.415122 7910 leader_election.cc:288] T 33a5319c9fda4ff78c160556c93520e3 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Requested vote from peers 7796ea3fba294d17a34386873969431d (127.22.45.65:43267), 2a9c1af8cf814030bde56b67ca5c2fbe (127.22.45.69:43887)
I20240502 20:46:40.415490 31493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33a5319c9fda4ff78c160556c93520e3" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe"
I20240502 20:46:40.415604 31493 raft_consensus.cc:3037] T 33a5319c9fda4ff78c160556c93520e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:40.415995 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33a5319c9fda4ff78c160556c93520e3" candidate_uuid: "09258ac095664fa99caf0707262cde13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:40.416105 29182 raft_consensus.cc:3037] T 33a5319c9fda4ff78c160556c93520e3 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:40.416582 31493 raft_consensus.cc:2454] T 33a5319c9fda4ff78c160556c93520e3 P 2a9c1af8cf814030bde56b67ca5c2fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:40.416837 29250 leader_election.cc:302] T 33a5319c9fda4ff78c160556c93520e3 P 09258ac095664fa99caf0707262cde13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, 2a9c1af8cf814030bde56b67ca5c2fbe; no voters:
I20240502 20:46:40.416972 29182 raft_consensus.cc:2454] T 33a5319c9fda4ff78c160556c93520e3 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09258ac095664fa99caf0707262cde13 in term 1.
I20240502 20:46:40.417034 29041 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=f5de73c60ecd401186da9a19d47b4698] alter complete (version 1)
I20240502 20:46:40.417165 7910 raft_consensus.cc:2781] T 33a5319c9fda4ff78c160556c93520e3 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:40.417256 7910 raft_consensus.cc:686] T 33a5319c9fda4ff78c160556c93520e3 P 09258ac095664fa99caf0707262cde13 [term 1 LEADER]: Becoming Leader. State: Replica: 09258ac095664fa99caf0707262cde13, State: Running, Role: LEADER
I20240502 20:46:40.417364 7910 consensus_queue.cc:237] T 33a5319c9fda4ff78c160556c93520e3 P 09258ac095664fa99caf0707262cde13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } }
W20240502 20:46:40.417703 29118 consensus_peers.cc:485] T b28fe6bccf4f48a6a5456e6ead5cc920 P 7796ea3fba294d17a34386873969431d -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 20:46:40.419493 29041 catalog_manager.cc:5484] T 33a5319c9fda4ff78c160556c93520e3 P 09258ac095664fa99caf0707262cde13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09258ac095664fa99caf0707262cde13 (127.22.45.66). New cstate: current_term: 1 leader_uuid: "09258ac095664fa99caf0707262cde13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 } } }
I20240502 20:46:40.419793 9082 tablet.cc:1722] T a891a7338cee4190bd2baca7e00a52e5 P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 20:46:40.420045 9082 tablet.cc:1620] T a891a7338cee4190bd2baca7e00a52e5 P 09258ac095664fa99caf0707262cde13: MemRowSet was empty: no flush needed.
W20240502 20:46:40.448196 31691 consensus_peers.cc:462] T 73c2e0de3fcf455ba1fecf701d23a05f P e7ab6b9053664ee5aa71c7978e26fe6a -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111)
W20240502 20:46:40.453963 29511 consensus_peers.cc:485] T b9c1ec018c544b3bb90fe57fe9995920 P db17f26a73f3447b92aa9a947e3a7057 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 20:46:40.474114 8680 raft_consensus.cc:482] T 86863ea03d0f4f52b7c49693fdd946f2 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:40.474202 8680 raft_consensus.cc:504] T 86863ea03d0f4f52b7c49693fdd946f2 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:40.474627 8680 leader_election.cc:288] T 86863ea03d0f4f52b7c49693fdd946f2 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:40.474987 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86863ea03d0f4f52b7c49693fdd946f2" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13" is_pre_election: true
I20240502 20:46:40.475122 29315 raft_consensus.cc:2454] T 86863ea03d0f4f52b7c49693fdd946f2 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:40.475222 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86863ea03d0f4f52b7c49693fdd946f2" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d" is_pre_election: true
I20240502 20:46:40.475354 29182 raft_consensus.cc:2454] T 86863ea03d0f4f52b7c49693fdd946f2 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:40.475529 31692 leader_election.cc:302] T 86863ea03d0f4f52b7c49693fdd946f2 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09258ac095664fa99caf0707262cde13, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:40.475701 8264 raft_consensus.cc:2781] T 86863ea03d0f4f52b7c49693fdd946f2 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:40.475759 8264 raft_consensus.cc:482] T 86863ea03d0f4f52b7c49693fdd946f2 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:40.475793 8264 raft_consensus.cc:3037] T 86863ea03d0f4f52b7c49693fdd946f2 P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:40.476610 8264 raft_consensus.cc:504] T 86863ea03d0f4f52b7c49693fdd946f2 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:40.476900 8264 leader_election.cc:288] T 86863ea03d0f4f52b7c49693fdd946f2 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Requested vote from peers 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945), 7796ea3fba294d17a34386873969431d (127.22.45.65:43267)
I20240502 20:46:40.477243 29182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86863ea03d0f4f52b7c49693fdd946f2" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7796ea3fba294d17a34386873969431d"
I20240502 20:46:40.477345 29182 raft_consensus.cc:3037] T 86863ea03d0f4f52b7c49693fdd946f2 P 7796ea3fba294d17a34386873969431d [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:40.477416 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86863ea03d0f4f52b7c49693fdd946f2" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09258ac095664fa99caf0707262cde13"
I20240502 20:46:40.477521 29315 raft_consensus.cc:3037] T 86863ea03d0f4f52b7c49693fdd946f2 P 09258ac095664fa99caf0707262cde13 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:40.478240 29182 raft_consensus.cc:2454] T 86863ea03d0f4f52b7c49693fdd946f2 P 7796ea3fba294d17a34386873969431d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:40.478569 31693 leader_election.cc:302] T 86863ea03d0f4f52b7c49693fdd946f2 P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7796ea3fba294d17a34386873969431d, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:40.478598 29315 raft_consensus.cc:2454] T 86863ea03d0f4f52b7c49693fdd946f2 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:40.478860 8264 raft_consensus.cc:2781] T 86863ea03d0f4f52b7c49693fdd946f2 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:40.478953 8264 raft_consensus.cc:686] T 86863ea03d0f4f52b7c49693fdd946f2 P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Becoming Leader. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Running, Role: LEADER
I20240502 20:46:40.479080 8264 consensus_queue.cc:237] T 86863ea03d0f4f52b7c49693fdd946f2 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } }
I20240502 20:46:40.481236 29041 catalog_manager.cc:5484] T 86863ea03d0f4f52b7c49693fdd946f2 P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: term changed from 0 to 1, leader changed from <none> to e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71). New cstate: current_term: 1 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 } } peers { permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 } } peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } }
W20240502 20:46:40.496966 29380 consensus_peers.cc:485] T 3c3bdc0b9e934ae2b81317a371817563 P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 315915efcf2b413ca7f9d874338e1b2d (127.22.45.70:46751): Couldn't send request to peer 315915efcf2b413ca7f9d874338e1b2d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.70:46751: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 20:46:40.519781 29041 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 20:46:40.524811 9080 tablet.cc:1722] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:40.525117 9080 tablet.cc:1620] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691: MemRowSet was empty: no flush needed.
I20240502 20:46:40.525584 9081 tablet.cc:1722] T a891a7338cee4190bd2baca7e00a52e5 P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:40.525897 9081 tablet.cc:1620] T a891a7338cee4190bd2baca7e00a52e5 P dbcdc6ac635b44b185c3eba63f66e9d6: MemRowSet was empty: no flush needed.
I20240502 20:46:40.527945 9082 tablet.cc:1722] T a891a7338cee4190bd2baca7e00a52e5 P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 20:46:40.528275 9082 tablet.cc:1620] T a891a7338cee4190bd2baca7e00a52e5 P 09258ac095664fa99caf0707262cde13: MemRowSet was empty: no flush needed.
I20240502 20:46:40.530413 29041 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table [id=f5de73c60ecd401186da9a19d47b4698] alter complete (version 2)
I20240502 20:46:40.632470 29042 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 20:46:40.637580 9080 tablet.cc:1722] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:40.638427 9081 tablet.cc:1722] T a891a7338cee4190bd2baca7e00a52e5 P dbcdc6ac635b44b185c3eba63f66e9d6: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:40.638258 9082 tablet.cc:1722] T a891a7338cee4190bd2baca7e00a52e5 P 09258ac095664fa99caf0707262cde13: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 20:46:40.645260 29042 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: rebalancer_dynamic_table_new_name [id=f5de73c60ecd401186da9a19d47b4698] alter complete (version 3)
I20240502 20:46:40.746416 29035 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:40.746582 29035 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60128:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 20:46:40.748701 29035 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 67913548da9f4872bd0a0732383a1f57: Sending DeleteTablet for 3 replicas of tablet a891a7338cee4190bd2baca7e00a52e5
I20240502 20:46:40.749208 31867 tablet_service.cc:1514] Processing DeleteTablet for tablet a891a7338cee4190bd2baca7e00a52e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:40 UTC) from {username='slave'} at 127.0.0.1:40510
I20240502 20:46:40.749284 29425 tablet_service.cc:1514] Processing DeleteTablet for tablet a891a7338cee4190bd2baca7e00a52e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:40 UTC) from {username='slave'} at 127.0.0.1:42668
I20240502 20:46:40.749568 29294 tablet_service.cc:1514] Processing DeleteTablet for tablet a891a7338cee4190bd2baca7e00a52e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 20:46:40 UTC) from {username='slave'} at 127.0.0.1:33250
I20240502 20:46:40.749864 9087 tablet_replica.cc:330] T a891a7338cee4190bd2baca7e00a52e5 P dbcdc6ac635b44b185c3eba63f66e9d6: stopping tablet replica
I20240502 20:46:40.749958 9086 tablet_replica.cc:330] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691: stopping tablet replica
I20240502 20:46:40.750077 9087 raft_consensus.cc:2229] T a891a7338cee4190bd2baca7e00a52e5 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:40.750195 9086 raft_consensus.cc:2229] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691 [term 1 LEADER]: Raft consensus shutting down.
I20240502 20:46:40.750247 9088 tablet_replica.cc:330] T a891a7338cee4190bd2baca7e00a52e5 P 09258ac095664fa99caf0707262cde13: stopping tablet replica
I20240502 20:46:40.750255 9087 raft_consensus.cc:2258] T a891a7338cee4190bd2baca7e00a52e5 P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:40.750432 9088 raft_consensus.cc:2229] T a891a7338cee4190bd2baca7e00a52e5 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 20:46:40.750479 9086 raft_consensus.cc:2258] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:40.750993 9088 raft_consensus.cc:2258] T a891a7338cee4190bd2baca7e00a52e5 P 09258ac095664fa99caf0707262cde13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 20:46:40.751067 9087 ts_tablet_manager.cc:1875] T a891a7338cee4190bd2baca7e00a52e5 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:40.751144 9086 ts_tablet_manager.cc:1875] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:40.751618 9088 ts_tablet_manager.cc:1875] T a891a7338cee4190bd2baca7e00a52e5 P 09258ac095664fa99caf0707262cde13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 20:46:40.752879 9087 ts_tablet_manager.cc:1888] T a891a7338cee4190bd2baca7e00a52e5 P dbcdc6ac635b44b185c3eba63f66e9d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:40.752944 9087 log.cc:1196] T a891a7338cee4190bd2baca7e00a52e5 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/wals/a891a7338cee4190bd2baca7e00a52e5
I20240502 20:46:40.753207 9087 ts_tablet_manager.cc:1909] T a891a7338cee4190bd2baca7e00a52e5 P dbcdc6ac635b44b185c3eba63f66e9d6: Deleting consensus metadata
I20240502 20:46:40.753387 9086 ts_tablet_manager.cc:1888] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:40.753468 9086 log.cc:1196] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/wals/a891a7338cee4190bd2baca7e00a52e5
I20240502 20:46:40.753746 9086 ts_tablet_manager.cc:1909] T a891a7338cee4190bd2baca7e00a52e5 P e6b60803d0764d228c863e2da65e3691: Deleting consensus metadata
I20240502 20:46:40.753784 29028 catalog_manager.cc:4830] TS dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095): tablet a891a7338cee4190bd2baca7e00a52e5 (table rebalancer_dynamic_table_new_name [id=f5de73c60ecd401186da9a19d47b4698]) successfully deleted
I20240502 20:46:40.753906 9088 ts_tablet_manager.cc:1888] T a891a7338cee4190bd2baca7e00a52e5 P 09258ac095664fa99caf0707262cde13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 20:46:40.753975 9088 log.cc:1196] T a891a7338cee4190bd2baca7e00a52e5 P 09258ac095664fa99caf0707262cde13: Deleting WAL directory at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/wals/a891a7338cee4190bd2baca7e00a52e5
I20240502 20:46:40.754388 29027 catalog_manager.cc:4830] TS e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207): tablet a891a7338cee4190bd2baca7e00a52e5 (table rebalancer_dynamic_table_new_name [id=f5de73c60ecd401186da9a19d47b4698]) successfully deleted
I20240502 20:46:40.754530 9088 ts_tablet_manager.cc:1909] T a891a7338cee4190bd2baca7e00a52e5 P 09258ac095664fa99caf0707262cde13: Deleting consensus metadata
I20240502 20:46:40.755079 29028 catalog_manager.cc:4830] TS 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): tablet a891a7338cee4190bd2baca7e00a52e5 (table rebalancer_dynamic_table_new_name [id=f5de73c60ecd401186da9a19d47b4698]) successfully deleted
I20240502 20:46:40.794703 8732 consensus_queue.cc:1035] T 33a5319c9fda4ff78c160556c93520e3 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9c1af8cf814030bde56b67ca5c2fbe" member_type: VOTER last_known_addr { host: "127.22.45.69" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:40.797252 8732 consensus_queue.cc:1035] T 33a5319c9fda4ff78c160556c93520e3 P 09258ac095664fa99caf0707262cde13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:40.816007 9061 consensus_queue.cc:1035] T ced8eced1cd04a52a292b4e96bbacc2b P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:40.818802 9061 consensus_queue.cc:1035] T ced8eced1cd04a52a292b4e96bbacc2b P e6b60803d0764d228c863e2da65e3691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:40.850452 29042 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60128:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 20:46:40.856371 29425 tablet_service.cc:1467] Processing CreateTablet for tablet c8a0232287e44b5fb301374af8561b6f (DEFAULT_TABLE table=rebalancer_dynamic_table [id=71960f1466744b1aa6e9b4cad1133a07]), partition=
I20240502 20:46:40.856649 29425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8a0232287e44b5fb301374af8561b6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:40.856817 31867 tablet_service.cc:1467] Processing CreateTablet for tablet c8a0232287e44b5fb301374af8561b6f (DEFAULT_TABLE table=rebalancer_dynamic_table [id=71960f1466744b1aa6e9b4cad1133a07]), partition=
I20240502 20:46:40.857020 31867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8a0232287e44b5fb301374af8561b6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:40.858747 9098 tablet_bootstrap.cc:492] T c8a0232287e44b5fb301374af8561b6f P dbcdc6ac635b44b185c3eba63f66e9d6: Bootstrap starting.
I20240502 20:46:40.859539 31736 tablet_service.cc:1467] Processing CreateTablet for tablet c8a0232287e44b5fb301374af8561b6f (DEFAULT_TABLE table=rebalancer_dynamic_table [id=71960f1466744b1aa6e9b4cad1133a07]), partition=
I20240502 20:46:40.859766 9098 tablet_bootstrap.cc:654] T c8a0232287e44b5fb301374af8561b6f P dbcdc6ac635b44b185c3eba63f66e9d6: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:40.859768 31736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8a0232287e44b5fb301374af8561b6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:40.860924 9098 tablet_bootstrap.cc:492] T c8a0232287e44b5fb301374af8561b6f P dbcdc6ac635b44b185c3eba63f66e9d6: No bootstrap required, opened a new log
I20240502 20:46:40.861002 9098 ts_tablet_manager.cc:1367] T c8a0232287e44b5fb301374af8561b6f P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:40.861647 9098 raft_consensus.cc:348] T c8a0232287e44b5fb301374af8561b6f P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:40.861740 9098 raft_consensus.cc:374] T c8a0232287e44b5fb301374af8561b6f P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:40.861760 9098 raft_consensus.cc:729] T c8a0232287e44b5fb301374af8561b6f P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcdc6ac635b44b185c3eba63f66e9d6, State: Initialized, Role: FOLLOWER
I20240502 20:46:40.861888 9098 consensus_queue.cc:260] T c8a0232287e44b5fb301374af8561b6f P dbcdc6ac635b44b185c3eba63f66e9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:40.862160 9098 ts_tablet_manager.cc:1398] T c8a0232287e44b5fb301374af8561b6f P dbcdc6ac635b44b185c3eba63f66e9d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:40.862371 9100 tablet_bootstrap.cc:492] T c8a0232287e44b5fb301374af8561b6f P e7ab6b9053664ee5aa71c7978e26fe6a: Bootstrap starting.
I20240502 20:46:40.863214 9100 tablet_bootstrap.cc:654] T c8a0232287e44b5fb301374af8561b6f P e7ab6b9053664ee5aa71c7978e26fe6a: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:40.868539 9102 tablet_bootstrap.cc:492] T c8a0232287e44b5fb301374af8561b6f P e6b60803d0764d228c863e2da65e3691: Bootstrap starting.
I20240502 20:46:40.869319 9102 tablet_bootstrap.cc:654] T c8a0232287e44b5fb301374af8561b6f P e6b60803d0764d228c863e2da65e3691: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:40.870338 9102 tablet_bootstrap.cc:492] T c8a0232287e44b5fb301374af8561b6f P e6b60803d0764d228c863e2da65e3691: No bootstrap required, opened a new log
I20240502 20:46:40.870419 9102 ts_tablet_manager.cc:1367] T c8a0232287e44b5fb301374af8561b6f P e6b60803d0764d228c863e2da65e3691: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:40.870832 9102 raft_consensus.cc:348] T c8a0232287e44b5fb301374af8561b6f P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:40.870929 9102 raft_consensus.cc:374] T c8a0232287e44b5fb301374af8561b6f P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:40.870958 9102 raft_consensus.cc:729] T c8a0232287e44b5fb301374af8561b6f P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b60803d0764d228c863e2da65e3691, State: Initialized, Role: FOLLOWER
I20240502 20:46:40.871091 9102 consensus_queue.cc:260] T c8a0232287e44b5fb301374af8561b6f P e6b60803d0764d228c863e2da65e3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:40.871349 9102 ts_tablet_manager.cc:1398] T c8a0232287e44b5fb301374af8561b6f P e6b60803d0764d228c863e2da65e3691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:40.874845 9100 tablet_bootstrap.cc:492] T c8a0232287e44b5fb301374af8561b6f P e7ab6b9053664ee5aa71c7978e26fe6a: No bootstrap required, opened a new log
I20240502 20:46:40.874929 9100 ts_tablet_manager.cc:1367] T c8a0232287e44b5fb301374af8561b6f P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20240502 20:46:40.875310 9100 raft_consensus.cc:348] T c8a0232287e44b5fb301374af8561b6f P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:40.875403 9100 raft_consensus.cc:374] T c8a0232287e44b5fb301374af8561b6f P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:40.875427 9100 raft_consensus.cc:729] T c8a0232287e44b5fb301374af8561b6f P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Initialized, Role: FOLLOWER
I20240502 20:46:40.875537 9100 consensus_queue.cc:260] T c8a0232287e44b5fb301374af8561b6f P e7ab6b9053664ee5aa71c7978e26fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:40.875782 9100 ts_tablet_manager.cc:1398] T c8a0232287e44b5fb301374af8561b6f P e7ab6b9053664ee5aa71c7978e26fe6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:40.889359 8264 consensus_queue.cc:1035] T 86863ea03d0f4f52b7c49693fdd946f2 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "09258ac095664fa99caf0707262cde13" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:40.893133 8680 consensus_queue.cc:1035] T 86863ea03d0f4f52b7c49693fdd946f2 P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7796ea3fba294d17a34386873969431d" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:40.904386 8680 raft_consensus.cc:482] T c8a0232287e44b5fb301374af8561b6f P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:40.904480 8680 raft_consensus.cc:504] T c8a0232287e44b5fb301374af8561b6f P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:40.904817 8680 leader_election.cc:288] T c8a0232287e44b5fb301374af8561b6f P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:40.905210 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a0232287e44b5fb301374af8561b6f" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691" is_pre_election: true
I20240502 20:46:40.905252 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a0232287e44b5fb301374af8561b6f" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" is_pre_election: true
I20240502 20:46:40.905331 31888 raft_consensus.cc:2454] T c8a0232287e44b5fb301374af8561b6f P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:40.905356 29446 raft_consensus.cc:2454] T c8a0232287e44b5fb301374af8561b6f P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 0.
I20240502 20:46:40.905625 31691 leader_election.cc:302] T c8a0232287e44b5fb301374af8561b6f P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e6b60803d0764d228c863e2da65e3691, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:40.905783 8680 raft_consensus.cc:2781] T c8a0232287e44b5fb301374af8561b6f P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:40.905831 8680 raft_consensus.cc:482] T c8a0232287e44b5fb301374af8561b6f P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:40.905879 8680 raft_consensus.cc:3037] T c8a0232287e44b5fb301374af8561b6f P e7ab6b9053664ee5aa71c7978e26fe6a [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:40.906934 8680 raft_consensus.cc:504] T c8a0232287e44b5fb301374af8561b6f P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:40.907218 8680 leader_election.cc:288] T c8a0232287e44b5fb301374af8561b6f P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Requested vote from peers dbcdc6ac635b44b185c3eba63f66e9d6 (127.22.45.67:44095), e6b60803d0764d228c863e2da65e3691 (127.22.45.72:35207)
I20240502 20:46:40.907641 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a0232287e44b5fb301374af8561b6f" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6"
I20240502 20:46:40.907745 29446 raft_consensus.cc:3037] T c8a0232287e44b5fb301374af8561b6f P dbcdc6ac635b44b185c3eba63f66e9d6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:40.908763 29446 raft_consensus.cc:2454] T c8a0232287e44b5fb301374af8561b6f P dbcdc6ac635b44b185c3eba63f66e9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:40.908953 31888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a0232287e44b5fb301374af8561b6f" candidate_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b60803d0764d228c863e2da65e3691"
I20240502 20:46:40.909039 31692 leader_election.cc:302] T c8a0232287e44b5fb301374af8561b6f P e7ab6b9053664ee5aa71c7978e26fe6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dbcdc6ac635b44b185c3eba63f66e9d6, e7ab6b9053664ee5aa71c7978e26fe6a; no voters:
I20240502 20:46:40.909061 31888 raft_consensus.cc:3037] T c8a0232287e44b5fb301374af8561b6f P e6b60803d0764d228c863e2da65e3691 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:40.909381 8680 raft_consensus.cc:2781] T c8a0232287e44b5fb301374af8561b6f P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:40.909456 8680 raft_consensus.cc:686] T c8a0232287e44b5fb301374af8561b6f P e7ab6b9053664ee5aa71c7978e26fe6a [term 1 LEADER]: Becoming Leader. State: Replica: e7ab6b9053664ee5aa71c7978e26fe6a, State: Running, Role: LEADER
I20240502 20:46:40.909559 8680 consensus_queue.cc:237] T c8a0232287e44b5fb301374af8561b6f P e7ab6b9053664ee5aa71c7978e26fe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } }
I20240502 20:46:40.909796 31888 raft_consensus.cc:2454] T c8a0232287e44b5fb301374af8561b6f P e6b60803d0764d228c863e2da65e3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ab6b9053664ee5aa71c7978e26fe6a in term 1.
I20240502 20:46:40.911628 29035 catalog_manager.cc:5484] T c8a0232287e44b5fb301374af8561b6f P e7ab6b9053664ee5aa71c7978e26fe6a reported cstate change: term changed from 0 to 1, leader changed from <none> to e7ab6b9053664ee5aa71c7978e26fe6a (127.22.45.71). New cstate: current_term: 1 leader_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ab6b9053664ee5aa71c7978e26fe6a" member_type: VOTER last_known_addr { host: "127.22.45.71" port: 36787 } } peers { permanent_uuid: "dbcdc6ac635b44b185c3eba63f66e9d6" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 44095 } } peers { permanent_uuid: "e6b60803d0764d228c863e2da65e3691" member_type: VOTER last_known_addr { host: "127.22.45.72" port: 35207 } } }
I20240502 20:46:40.919737 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 29102
W20240502 20:46:40.928467 29382 connection.cc:535] client connection to 127.22.45.65:43267 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20240502 20:46:40.928802 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 29233
W20240502 20:46:40.929780 29382 consensus_peers.cc:485] T 3126707d1c8c4c1786da5bfdfce53eb0 P dbcdc6ac635b44b185c3eba63f66e9d6 -> Peer 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): Couldn't send request to peer 7796ea3fba294d17a34386873969431d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.65:43267: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:40.933035 31824 consensus_peers.cc:485] T 7a8b04cb606f423f8dd1d489d47408d2 P e6b60803d0764d228c863e2da65e3691 -> Peer 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): Couldn't send request to peer 7796ea3fba294d17a34386873969431d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.65:43267: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 20:46:40.936631 31823 connection.cc:535] client connection to 127.22.45.66:40945 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20240502 20:46:40.937323 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 29364
W20240502 20:46:40.938637 31823 consensus_peers.cc:485] T 6b675480162c4a9799db51ea4b6504a5 P e6b60803d0764d228c863e2da65e3691 -> Peer 09258ac095664fa99caf0707262cde13 (127.22.45.66:40945): Couldn't send request to peer 09258ac095664fa99caf0707262cde13. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.66:40945: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:40.949882 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 29495
W20240502 20:46:40.957142 31431 consensus_peers.cc:485] T 21ee3d7c6f764d0483b335da40e61efc P 2a9c1af8cf814030bde56b67ca5c2fbe -> Peer 7796ea3fba294d17a34386873969431d (127.22.45.65:43267): Couldn't send request to peer 7796ea3fba294d17a34386873969431d. Status: Network error: Client connection negotiation failed: client connection to 127.22.45.65:43267: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 20:46:40.958501 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 31413
I20240502 20:46:40.965867 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 31675
I20240502 20:46:40.972527 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 31806
I20240502 20:46:40.979019 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 29011
2024-05-02T20:46:40Z chronyd exiting
I20240502 20:46:40.995165 22709 test_util.cc:175] -----------------------------------------------
I20240502 20:46:40.995206 22709 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714682725224411-22709-0
[ FAILED ] DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing/0, where GetParam() = 4-byte object <00-00 00-00> (46552 ms)
[----------] 1 test from DDLDuringRebalancingTest (46552 ms total)
[----------] 1 test from RebalancingDuringElectionStormTest
[ RUN ] RebalancingDuringElectionStormTest.RoundRobin/0
I20240502 20:46:40.995887 22709 test_util.cc:268] Using random seed: -1287553493
I20240502 20:46:40.996683 22709 ts_itest-base.cc:112] Starting cluster with:
I20240502 20:46:40.996727 22709 ts_itest-base.cc:113] --------------
I20240502 20:46:40.996737 22709 ts_itest-base.cc:114] 8 tablet servers
I20240502 20:46:40.996747 22709 ts_itest-base.cc:115] 3 replicas per TS
I20240502 20:46:40.996759 22709 ts_itest-base.cc:116] --------------
2024-05-02T20:46:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T20:46:40Z Disabled control of system clock
I20240502 20:46:41.004559 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.45.126:41377
--webserver_interface=127.22.45.126
--webserver_port=0
--builtin_ntp_servers=127.22.45.84:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.45.126:41377
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=false
--enable_range_replica_placement=false with env {}
W20240502 20:46:41.103399 9114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:41.103583 9114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:46:41.103621 9114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:41.106472 9114 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:46:41.106524 9114 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 20:46:41.106549 9114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:46:41.106570 9114 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 20:46:41.106586 9114 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 20:46:41.109982 9114 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.22.45.126:41377
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.45.126:41377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.22.45.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.9114
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:46:41.110692 9114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:46:41.111651 9114 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:46:41.118347 9122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:41.118731 9120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:41.119114 9119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:46:41.119616 9114 server_base.cc:1034] running on GCE node
I20240502 20:46:41.120029 9114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:46:41.120816 9114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:46:41.121958 9114 hybrid_clock.cc:648] HybridClock initialized: now 1714682801121947 us; error 24 us; skew 500 ppm
I20240502 20:46:41.123620 9114 webserver.cc:460] Webserver started at http://127.22.45.126:34163/ using document root <none> and password file <none>
I20240502 20:46:41.124094 9114 fs_manager.cc:359] Metadata directory not provided
I20240502 20:46:41.124143 9114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:46:41.124305 9114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:46:41.125833 9114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "3165faf23f704f8489bacda4c220fbe4"
format_stamp: "Formatted at 2024-05-02 20:46:41 on dist-test-slave-g7g5"
I20240502 20:46:41.126253 9114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "3165faf23f704f8489bacda4c220fbe4"
format_stamp: "Formatted at 2024-05-02 20:46:41 on dist-test-slave-g7g5"
I20240502 20:46:41.129283 9114 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:41.131325 9128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:41.132236 9114 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:41.132344 9114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal
uuid: "3165faf23f704f8489bacda4c220fbe4"
format_stamp: "Formatted at 2024-05-02 20:46:41 on dist-test-slave-g7g5"
I20240502 20:46:41.132422 9114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:46:41.143401 9114 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:46:41.143926 9114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:46:41.144066 9114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:46:41.149941 9114 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.126:41377
I20240502 20:46:41.149947 9180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.126:41377 every 8 connection(s)
I20240502 20:46:41.150790 9114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 20:46:41.153053 9181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:41.157523 9181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3165faf23f704f8489bacda4c220fbe4: Bootstrap starting.
I20240502 20:46:41.158375 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 9114
I20240502 20:46:41.158514 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 20:46:41.159896 9181 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3165faf23f704f8489bacda4c220fbe4: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:41.160498 9181 log.cc:824] T 00000000000000000000000000000000 P 3165faf23f704f8489bacda4c220fbe4: Log is configured to *not* fsync() on all Append() calls
I20240502 20:46:41.162035 9181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3165faf23f704f8489bacda4c220fbe4: No bootstrap required, opened a new log
I20240502 20:46:41.164516 9181 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3165faf23f704f8489bacda4c220fbe4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3165faf23f704f8489bacda4c220fbe4" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 41377 } }
I20240502 20:46:41.164678 9181 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3165faf23f704f8489bacda4c220fbe4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:41.164737 9181 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3165faf23f704f8489bacda4c220fbe4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3165faf23f704f8489bacda4c220fbe4, State: Initialized, Role: FOLLOWER
I20240502 20:46:41.165128 9181 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3165faf23f704f8489bacda4c220fbe4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3165faf23f704f8489bacda4c220fbe4" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 41377 } }
I20240502 20:46:41.165273 9181 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3165faf23f704f8489bacda4c220fbe4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 20:46:41.165333 9181 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3165faf23f704f8489bacda4c220fbe4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 20:46:41.165446 9181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3165faf23f704f8489bacda4c220fbe4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:41.166152 9181 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3165faf23f704f8489bacda4c220fbe4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3165faf23f704f8489bacda4c220fbe4" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 41377 } }
I20240502 20:46:41.166491 9181 leader_election.cc:302] T 00000000000000000000000000000000 P 3165faf23f704f8489bacda4c220fbe4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3165faf23f704f8489bacda4c220fbe4; no voters:
I20240502 20:46:41.166777 9181 leader_election.cc:288] T 00000000000000000000000000000000 P 3165faf23f704f8489bacda4c220fbe4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 20:46:41.166886 9186 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3165faf23f704f8489bacda4c220fbe4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:41.167440 9186 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3165faf23f704f8489bacda4c220fbe4 [term 1 LEADER]: Becoming Leader. State: Replica: 3165faf23f704f8489bacda4c220fbe4, State: Running, Role: LEADER
I20240502 20:46:41.167662 9181 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3165faf23f704f8489bacda4c220fbe4 [sys.catalog]: configured and running, proceeding with master startup.
I20240502 20:46:41.167698 9186 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3165faf23f704f8489bacda4c220fbe4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3165faf23f704f8489bacda4c220fbe4" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 41377 } }
I20240502 20:46:41.169164 9188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3165faf23f704f8489bacda4c220fbe4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3165faf23f704f8489bacda4c220fbe4. Latest consensus state: current_term: 1 leader_uuid: "3165faf23f704f8489bacda4c220fbe4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3165faf23f704f8489bacda4c220fbe4" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 41377 } } }
I20240502 20:46:41.169173 9187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3165faf23f704f8489bacda4c220fbe4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3165faf23f704f8489bacda4c220fbe4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3165faf23f704f8489bacda4c220fbe4" member_type: VOTER last_known_addr { host: "127.22.45.126" port: 41377 } } }
I20240502 20:46:41.169288 9188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3165faf23f704f8489bacda4c220fbe4 [sys.catalog]: This master's current role is: LEADER
I20240502 20:46:41.169288 9187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3165faf23f704f8489bacda4c220fbe4 [sys.catalog]: This master's current role is: LEADER
W20240502 20:46:41.171734 9202 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3165faf23f704f8489bacda4c220fbe4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240502 20:46:41.171797 9202 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20240502 20:46:41.172067 9200 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 20:46:41.172981 9200 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 20:46:41.177610 9200 catalog_manager.cc:1347] Generated new cluster ID: 8c573e64a47742718377ab6b270506e7
I20240502 20:46:41.177659 9200 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 20:46:41.195312 9200 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 20:46:41.196110 9200 catalog_manager.cc:1504] Loading token signing keys...
I20240502 20:46:41.202055 9200 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 3165faf23f704f8489bacda4c220fbe4: Generated new TSK 0
I20240502 20:46:41.202615 9200 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 20:46:41.206622 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.65:0
--local_ip_for_outbound_sockets=127.22.45.65
--webserver_interface=127.22.45.65
--webserver_port=0
--tserver_master_addrs=127.22.45.126:41377
--builtin_ntp_servers=127.22.45.84:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 20:46:41.306207 9206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:41.306399 9206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:46:41.306488 9206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:41.309363 9206 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:46:41.309422 9206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:46:41.309504 9206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.65
I20240502 20:46:41.312961 9206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.22.45.65
--webserver_port=0
--tserver_master_addrs=127.22.45.126:41377
--never_fsync=true
--heap_profile_path=/tmp/kudu.9206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.65
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:46:41.313709 9206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:46:41.314766 9206 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:46:41.322327 9211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:41.322862 9212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:41.323271 9214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:46:41.323707 9206 server_base.cc:1034] running on GCE node
I20240502 20:46:41.324031 9206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:46:41.324501 9206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:46:41.325645 9206 hybrid_clock.cc:648] HybridClock initialized: now 1714682801325632 us; error 27 us; skew 500 ppm
I20240502 20:46:41.327315 9206 webserver.cc:460] Webserver started at http://127.22.45.65:44311/ using document root <none> and password file <none>
I20240502 20:46:41.327798 9206 fs_manager.cc:359] Metadata directory not provided
I20240502 20:46:41.327854 9206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:46:41.328037 9206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:46:41.329535 9206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
format_stamp: "Formatted at 2024-05-02 20:46:41 on dist-test-slave-g7g5"
I20240502 20:46:41.329972 9206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
format_stamp: "Formatted at 2024-05-02 20:46:41 on dist-test-slave-g7g5"
I20240502 20:46:41.332975 9206 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 20:46:41.334887 9220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:41.335803 9206 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:41.335916 9206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
format_stamp: "Formatted at 2024-05-02 20:46:41 on dist-test-slave-g7g5"
I20240502 20:46:41.335990 9206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:46:41.348338 9206 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:46:41.348819 9206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:46:41.348956 9206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:46:41.349397 9206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:46:41.350178 9206 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:46:41.350224 9206 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:41.350270 9206 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:46:41.350297 9206 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:41.357831 9206 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.65:38355
I20240502 20:46:41.357877 9336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.65:38355 every 8 connection(s)
I20240502 20:46:41.358664 9206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 20:46:41.361477 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 9206
I20240502 20:46:41.361588 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 20:46:41.363476 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.66:0
--local_ip_for_outbound_sockets=127.22.45.66
--webserver_interface=127.22.45.66
--webserver_port=0
--tserver_master_addrs=127.22.45.126:41377
--builtin_ntp_servers=127.22.45.84:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:46:41.366776 9337 heartbeater.cc:347] Connected to a master server at 127.22.45.126:41377
I20240502 20:46:41.367008 9337 heartbeater.cc:464] Registering TS with master...
I20240502 20:46:41.367497 9337 heartbeater.cc:511] Master 127.22.45.126:41377 requested a full tablet report, sending...
I20240502 20:46:41.368865 9145 ts_manager.cc:194] Registered new tserver with Master: 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:41.370028 9145 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.65:37461
W20240502 20:46:41.462814 9340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:41.462997 9340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:46:41.463054 9340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:41.465901 9340 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:46:41.465960 9340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:46:41.466041 9340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.66
I20240502 20:46:41.469548 9340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.22.45.66
--webserver_port=0
--tserver_master_addrs=127.22.45.126:41377
--never_fsync=true
--heap_profile_path=/tmp/kudu.9340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.66
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:46:41.470263 9340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:46:41.471241 9340 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20240502 20:46:41.478899 9340 server_base.cc:1034] running on GCE node
W20240502 20:46:41.479245 9345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:41.479573 9348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:41.479614 9346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:46:41.479951 9340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:46:41.480423 9340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:46:41.481576 9340 hybrid_clock.cc:648] HybridClock initialized: now 1714682801481554 us; error 36 us; skew 500 ppm
I20240502 20:46:41.483199 9340 webserver.cc:460] Webserver started at http://127.22.45.66:34193/ using document root <none> and password file <none>
I20240502 20:46:41.483678 9340 fs_manager.cc:359] Metadata directory not provided
I20240502 20:46:41.483733 9340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:46:41.483883 9340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:46:41.485323 9340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "88a19090795d47c2a1ea1dd979fb62bc"
format_stamp: "Formatted at 2024-05-02 20:46:41 on dist-test-slave-g7g5"
I20240502 20:46:41.485756 9340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "88a19090795d47c2a1ea1dd979fb62bc"
format_stamp: "Formatted at 2024-05-02 20:46:41 on dist-test-slave-g7g5"
I20240502 20:46:41.488730 9340 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20240502 20:46:41.490638 9354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:41.491518 9340 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20240502 20:46:41.491622 9340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "88a19090795d47c2a1ea1dd979fb62bc"
format_stamp: "Formatted at 2024-05-02 20:46:41 on dist-test-slave-g7g5"
I20240502 20:46:41.491695 9340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:46:41.505184 9340 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:46:41.505637 9340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:46:41.505770 9340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:46:41.506201 9340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:46:41.506995 9340 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:46:41.507040 9340 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:41.507088 9340 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:46:41.507117 9340 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:41.514706 9340 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.66:46815
I20240502 20:46:41.514748 9471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.66:46815 every 8 connection(s)
I20240502 20:46:41.515475 9340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 20:46:41.518344 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 9340
I20240502 20:46:41.518496 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 20:46:41.520639 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.67:0
--local_ip_for_outbound_sockets=127.22.45.67
--webserver_interface=127.22.45.67
--webserver_port=0
--tserver_master_addrs=127.22.45.126:41377
--builtin_ntp_servers=127.22.45.84:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:46:41.523595 9472 heartbeater.cc:347] Connected to a master server at 127.22.45.126:41377
I20240502 20:46:41.523785 9472 heartbeater.cc:464] Registering TS with master...
I20240502 20:46:41.524183 9472 heartbeater.cc:511] Master 127.22.45.126:41377 requested a full tablet report, sending...
I20240502 20:46:41.525040 9145 ts_manager.cc:194] Registered new tserver with Master: 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:41.525557 9145 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.66:35735
W20240502 20:46:41.620451 9475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:41.620627 9475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:46:41.620672 9475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:41.623561 9475 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:46:41.623616 9475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:46:41.623699 9475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.67
I20240502 20:46:41.627104 9475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.22.45.67
--webserver_port=0
--tserver_master_addrs=127.22.45.126:41377
--never_fsync=true
--heap_profile_path=/tmp/kudu.9475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.67
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:46:41.627848 9475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:46:41.628852 9475 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:46:41.636560 9480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:41.636809 9483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:41.637387 9481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:46:41.637815 9475 server_base.cc:1034] running on GCE node
I20240502 20:46:41.638129 9475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:46:41.638643 9475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:46:41.639782 9475 hybrid_clock.cc:648] HybridClock initialized: now 1714682801639768 us; error 28 us; skew 500 ppm
I20240502 20:46:41.641423 9475 webserver.cc:460] Webserver started at http://127.22.45.67:33895/ using document root <none> and password file <none>
I20240502 20:46:41.641904 9475 fs_manager.cc:359] Metadata directory not provided
I20240502 20:46:41.641960 9475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:46:41.642117 9475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:46:41.643625 9475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
format_stamp: "Formatted at 2024-05-02 20:46:41 on dist-test-slave-g7g5"
I20240502 20:46:41.644060 9475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
format_stamp: "Formatted at 2024-05-02 20:46:41 on dist-test-slave-g7g5"
I20240502 20:46:41.647115 9475 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 20:46:41.648975 9489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:41.649837 9475 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:41.649930 9475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
format_stamp: "Formatted at 2024-05-02 20:46:41 on dist-test-slave-g7g5"
I20240502 20:46:41.650002 9475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:46:41.663848 9475 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:46:41.664302 9475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:46:41.664433 9475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:46:41.664865 9475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:46:41.665647 9475 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:46:41.665692 9475 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:41.665740 9475 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:46:41.665766 9475 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:41.674139 9475 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.67:33617
I20240502 20:46:41.674161 9602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.67:33617 every 8 connection(s)
I20240502 20:46:41.674968 9475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 20:46:41.675840 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 9475
I20240502 20:46:41.675933 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 20:46:41.678530 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.68:0
--local_ip_for_outbound_sockets=127.22.45.68
--webserver_interface=127.22.45.68
--webserver_port=0
--tserver_master_addrs=127.22.45.126:41377
--builtin_ntp_servers=127.22.45.84:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:46:41.683509 9603 heartbeater.cc:347] Connected to a master server at 127.22.45.126:41377
I20240502 20:46:41.683728 9603 heartbeater.cc:464] Registering TS with master...
I20240502 20:46:41.684214 9603 heartbeater.cc:511] Master 127.22.45.126:41377 requested a full tablet report, sending...
I20240502 20:46:41.685107 9145 ts_manager.cc:194] Registered new tserver with Master: 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:41.685609 9145 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.67:51267
W20240502 20:46:41.780072 9606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:41.780303 9606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:46:41.780359 9606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:41.784001 9606 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:46:41.784062 9606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:46:41.784123 9606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.68
I20240502 20:46:41.787508 9606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.22.45.68
--webserver_port=0
--tserver_master_addrs=127.22.45.126:41377
--never_fsync=true
--heap_profile_path=/tmp/kudu.9606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.68
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:46:41.788201 9606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:46:41.789148 9606 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:46:41.796660 9614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:41.796988 9611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:41.797248 9612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:46:41.797721 9606 server_base.cc:1034] running on GCE node
I20240502 20:46:41.798035 9606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:46:41.798518 9606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:46:41.799654 9606 hybrid_clock.cc:648] HybridClock initialized: now 1714682801799640 us; error 26 us; skew 500 ppm
I20240502 20:46:41.801337 9606 webserver.cc:460] Webserver started at http://127.22.45.68:33611/ using document root <none> and password file <none>
I20240502 20:46:41.801823 9606 fs_manager.cc:359] Metadata directory not provided
I20240502 20:46:41.801877 9606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:46:41.802035 9606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:46:41.803540 9606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "64136bd82a30486e9d56fc28da4bd448"
format_stamp: "Formatted at 2024-05-02 20:46:41 on dist-test-slave-g7g5"
I20240502 20:46:41.803966 9606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "64136bd82a30486e9d56fc28da4bd448"
format_stamp: "Formatted at 2024-05-02 20:46:41 on dist-test-slave-g7g5"
I20240502 20:46:41.806914 9606 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 20:46:41.808816 9620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:41.809680 9606 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:41.809775 9606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "64136bd82a30486e9d56fc28da4bd448"
format_stamp: "Formatted at 2024-05-02 20:46:41 on dist-test-slave-g7g5"
I20240502 20:46:41.809847 9606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:46:41.845623 9606 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:46:41.846117 9606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:46:41.846252 9606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:46:41.846753 9606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:46:41.847548 9606 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:46:41.847592 9606 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:41.847642 9606 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:46:41.847667 9606 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:41.855562 9606 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.68:42225
I20240502 20:46:41.855589 9734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.68:42225 every 8 connection(s)
I20240502 20:46:41.856345 9606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 20:46:41.863550 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 9606
I20240502 20:46:41.863677 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 20:46:41.863901 9735 heartbeater.cc:347] Connected to a master server at 127.22.45.126:41377
I20240502 20:46:41.864104 9735 heartbeater.cc:464] Registering TS with master...
I20240502 20:46:41.864502 9735 heartbeater.cc:511] Master 127.22.45.126:41377 requested a full tablet report, sending...
I20240502 20:46:41.865381 9145 ts_manager.cc:194] Registered new tserver with Master: 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:41.865624 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.69:0
--local_ip_for_outbound_sockets=127.22.45.69
--webserver_interface=127.22.45.69
--webserver_port=0
--tserver_master_addrs=127.22.45.126:41377
--builtin_ntp_servers=127.22.45.84:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:46:41.865913 9145 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.68:52913
W20240502 20:46:41.963827 9738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:41.964020 9738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:46:41.964076 9738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:41.966928 9738 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:46:41.966991 9738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:46:41.967070 9738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.69
I20240502 20:46:41.970504 9738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.69:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.22.45.69
--webserver_port=0
--tserver_master_addrs=127.22.45.126:41377
--never_fsync=true
--heap_profile_path=/tmp/kudu.9738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.69
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:46:41.971185 9738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:46:41.972119 9738 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:46:41.979323 9746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:41.979673 9743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:41.980026 9744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:46:41.980443 9738 server_base.cc:1034] running on GCE node
I20240502 20:46:41.980769 9738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:46:41.981232 9738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:46:41.982383 9738 hybrid_clock.cc:648] HybridClock initialized: now 1714682801982368 us; error 28 us; skew 500 ppm
I20240502 20:46:41.984027 9738 webserver.cc:460] Webserver started at http://127.22.45.69:44027/ using document root <none> and password file <none>
I20240502 20:46:41.984499 9738 fs_manager.cc:359] Metadata directory not provided
I20240502 20:46:41.984555 9738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:46:41.984704 9738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:46:41.986131 9738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "af8c170629d044fead8b1e8f948cbeee"
format_stamp: "Formatted at 2024-05-02 20:46:41 on dist-test-slave-g7g5"
I20240502 20:46:41.986603 9738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "af8c170629d044fead8b1e8f948cbeee"
format_stamp: "Formatted at 2024-05-02 20:46:41 on dist-test-slave-g7g5"
I20240502 20:46:41.989427 9738 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:41.991392 9752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:41.992236 9738 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:41.992329 9738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "af8c170629d044fead8b1e8f948cbeee"
format_stamp: "Formatted at 2024-05-02 20:46:41 on dist-test-slave-g7g5"
I20240502 20:46:41.992404 9738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:46:42.004755 9738 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:46:42.005196 9738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:46:42.005326 9738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:46:42.005746 9738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:46:42.006537 9738 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:46:42.006583 9738 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:42.006630 9738 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:46:42.006655 9738 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:42.015678 9738 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.69:41807
I20240502 20:46:42.015718 9866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.69:41807 every 8 connection(s)
I20240502 20:46:42.016435 9738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 20:46:42.020175 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 9738
I20240502 20:46:42.020298 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240502 20:46:42.022433 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.70:0
--local_ip_for_outbound_sockets=127.22.45.70
--webserver_interface=127.22.45.70
--webserver_port=0
--tserver_master_addrs=127.22.45.126:41377
--builtin_ntp_servers=127.22.45.84:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:46:42.024652 9867 heartbeater.cc:347] Connected to a master server at 127.22.45.126:41377
I20240502 20:46:42.024830 9867 heartbeater.cc:464] Registering TS with master...
I20240502 20:46:42.025198 9867 heartbeater.cc:511] Master 127.22.45.126:41377 requested a full tablet report, sending...
I20240502 20:46:42.026047 9145 ts_manager.cc:194] Registered new tserver with Master: af8c170629d044fead8b1e8f948cbeee (127.22.45.69:41807)
I20240502 20:46:42.026767 9145 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.69:59925
W20240502 20:46:42.120671 9870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:42.120846 9870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:46:42.120885 9870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:42.123736 9870 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:46:42.123786 9870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:46:42.123852 9870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.70
I20240502 20:46:42.127427 9870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.70:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.22.45.70
--webserver_port=0
--tserver_master_addrs=127.22.45.126:41377
--never_fsync=true
--heap_profile_path=/tmp/kudu.9870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.70
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:46:42.128114 9870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:46:42.129055 9870 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:46:42.136502 9878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:42.136759 9875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:42.137135 9876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:46:42.137528 9870 server_base.cc:1034] running on GCE node
I20240502 20:46:42.137816 9870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:46:42.138247 9870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:46:42.139392 9870 hybrid_clock.cc:648] HybridClock initialized: now 1714682802139359 us; error 44 us; skew 500 ppm
I20240502 20:46:42.141014 9870 webserver.cc:460] Webserver started at http://127.22.45.70:35361/ using document root <none> and password file <none>
I20240502 20:46:42.141470 9870 fs_manager.cc:359] Metadata directory not provided
I20240502 20:46:42.141520 9870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:46:42.141677 9870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:46:42.143189 9870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "e17061864e514564ba48c05dd80f7254"
format_stamp: "Formatted at 2024-05-02 20:46:42 on dist-test-slave-g7g5"
I20240502 20:46:42.143602 9870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "e17061864e514564ba48c05dd80f7254"
format_stamp: "Formatted at 2024-05-02 20:46:42 on dist-test-slave-g7g5"
I20240502 20:46:42.146534 9870 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 20:46:42.148445 9884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:42.149324 9870 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:42.149418 9870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "e17061864e514564ba48c05dd80f7254"
format_stamp: "Formatted at 2024-05-02 20:46:42 on dist-test-slave-g7g5"
I20240502 20:46:42.149493 9870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:46:42.162791 9870 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:46:42.163242 9870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:46:42.163370 9870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:46:42.163785 9870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:46:42.164520 9870 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:46:42.164564 9870 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:42.164609 9870 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:46:42.164636 9870 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:42.172775 9870 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.70:37255
I20240502 20:46:42.172798 9998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.70:37255 every 8 connection(s)
I20240502 20:46:42.173532 9870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 20:46:42.177250 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 9870
I20240502 20:46:42.177354 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240502 20:46:42.179221 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.71:0
--local_ip_for_outbound_sockets=127.22.45.71
--webserver_interface=127.22.45.71
--webserver_port=0
--tserver_master_addrs=127.22.45.126:41377
--builtin_ntp_servers=127.22.45.84:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:46:42.181309 9999 heartbeater.cc:347] Connected to a master server at 127.22.45.126:41377
I20240502 20:46:42.181489 9999 heartbeater.cc:464] Registering TS with master...
I20240502 20:46:42.181851 9999 heartbeater.cc:511] Master 127.22.45.126:41377 requested a full tablet report, sending...
I20240502 20:46:42.182690 9145 ts_manager.cc:194] Registered new tserver with Master: e17061864e514564ba48c05dd80f7254 (127.22.45.70:37255)
I20240502 20:46:42.183192 9145 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.70:34193
W20240502 20:46:42.281006 10002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:42.281179 10002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:46:42.281224 10002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:42.284139 10002 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:46:42.284189 10002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:46:42.284266 10002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.71
I20240502 20:46:42.287667 10002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.71:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.22.45.71
--webserver_port=0
--tserver_master_addrs=127.22.45.126:41377
--never_fsync=true
--heap_profile_path=/tmp/kudu.10002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.71
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:46:42.288349 10002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:46:42.289289 10002 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:46:42.296717 10010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:42.296973 10007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:42.297300 10008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:46:42.297760 10002 server_base.cc:1034] running on GCE node
I20240502 20:46:42.298086 10002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:46:42.298570 10002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:46:42.299708 10002 hybrid_clock.cc:648] HybridClock initialized: now 1714682802299691 us; error 36 us; skew 500 ppm
I20240502 20:46:42.301332 10002 webserver.cc:460] Webserver started at http://127.22.45.71:33945/ using document root <none> and password file <none>
I20240502 20:46:42.301795 10002 fs_manager.cc:359] Metadata directory not provided
I20240502 20:46:42.301852 10002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:46:42.302009 10002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:46:42.303478 10002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "d22b71ae7ed94ce295da0ca9955444d8"
format_stamp: "Formatted at 2024-05-02 20:46:42 on dist-test-slave-g7g5"
I20240502 20:46:42.303900 10002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "d22b71ae7ed94ce295da0ca9955444d8"
format_stamp: "Formatted at 2024-05-02 20:46:42 on dist-test-slave-g7g5"
I20240502 20:46:42.306787 10002 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 20:46:42.308684 10016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:42.309530 10002 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:42.309625 10002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "d22b71ae7ed94ce295da0ca9955444d8"
format_stamp: "Formatted at 2024-05-02 20:46:42 on dist-test-slave-g7g5"
I20240502 20:46:42.309696 10002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:46:42.320497 10002 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:46:42.320935 10002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:46:42.321069 10002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:46:42.321489 10002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:46:42.322238 10002 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:46:42.322284 10002 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:42.322329 10002 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:46:42.322355 10002 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:42.329743 10002 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.71:44533
I20240502 20:46:42.329792 10132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.71:44533 every 8 connection(s)
I20240502 20:46:42.330544 10002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 20:46:42.334079 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 10002
I20240502 20:46:42.334198 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240502 20:46:42.336380 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.72:0
--local_ip_for_outbound_sockets=127.22.45.72
--webserver_interface=127.22.45.72
--webserver_port=0
--tserver_master_addrs=127.22.45.126:41377
--builtin_ntp_servers=127.22.45.84:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:46:42.338868 10133 heartbeater.cc:347] Connected to a master server at 127.22.45.126:41377
I20240502 20:46:42.339056 10133 heartbeater.cc:464] Registering TS with master...
I20240502 20:46:42.339422 10133 heartbeater.cc:511] Master 127.22.45.126:41377 requested a full tablet report, sending...
I20240502 20:46:42.340245 9145 ts_manager.cc:194] Registered new tserver with Master: d22b71ae7ed94ce295da0ca9955444d8 (127.22.45.71:44533)
I20240502 20:46:42.340741 9145 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.71:48701
I20240502 20:46:42.372330 9337 heartbeater.cc:503] Master 127.22.45.126:41377 was elected leader, sending a full tablet report...
W20240502 20:46:42.438511 10136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:42.438709 10136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:46:42.438763 10136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:42.441780 10136 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:46:42.441848 10136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:46:42.441944 10136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.72
I20240502 20:46:42.445336 10136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.72:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.22.45.72
--webserver_port=0
--tserver_master_addrs=127.22.45.126:41377
--never_fsync=true
--heap_profile_path=/tmp/kudu.10136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.72
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:46:42.446040 10136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:46:42.447046 10136 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:46:42.454720 10141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:42.455042 10142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:42.455417 10144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:46:42.455849 10136 server_base.cc:1034] running on GCE node
I20240502 20:46:42.456152 10136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:46:42.456590 10136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:46:42.457739 10136 hybrid_clock.cc:648] HybridClock initialized: now 1714682802457730 us; error 28 us; skew 500 ppm
I20240502 20:46:42.459390 10136 webserver.cc:460] Webserver started at http://127.22.45.72:42565/ using document root <none> and password file <none>
I20240502 20:46:42.459857 10136 fs_manager.cc:359] Metadata directory not provided
I20240502 20:46:42.459913 10136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:46:42.460064 10136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 20:46:42.461500 10136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "e1107bea1a174997a62341f2c771f268"
format_stamp: "Formatted at 2024-05-02 20:46:42 on dist-test-slave-g7g5"
I20240502 20:46:42.461916 10136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "e1107bea1a174997a62341f2c771f268"
format_stamp: "Formatted at 2024-05-02 20:46:42 on dist-test-slave-g7g5"
I20240502 20:46:42.464818 10136 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 20:46:42.466660 10151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:42.467554 10136 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:42.467655 10136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "e1107bea1a174997a62341f2c771f268"
format_stamp: "Formatted at 2024-05-02 20:46:42 on dist-test-slave-g7g5"
I20240502 20:46:42.467728 10136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:46:42.478034 10136 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:46:42.478541 10136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:46:42.478667 10136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:46:42.479080 10136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:46:42.479808 10136 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:46:42.479852 10136 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:42.479898 10136 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:46:42.479923 10136 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:42.487246 10136 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.72:43571
I20240502 20:46:42.487269 10265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.72:43571 every 8 connection(s)
I20240502 20:46:42.487991 10136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 20:46:42.491636 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 10136
I20240502 20:46:42.491746 22709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240502 20:46:42.495903 10266 heartbeater.cc:347] Connected to a master server at 127.22.45.126:41377
I20240502 20:46:42.496099 10266 heartbeater.cc:464] Registering TS with master...
I20240502 20:46:42.496506 10266 heartbeater.cc:511] Master 127.22.45.126:41377 requested a full tablet report, sending...
I20240502 20:46:42.497330 9145 ts_manager.cc:194] Registered new tserver with Master: e1107bea1a174997a62341f2c771f268 (127.22.45.72:43571)
I20240502 20:46:42.497843 9145 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.72:50973
I20240502 20:46:42.505251 22709 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240502 20:46:42.514238 22709 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240502 20:46:42.514305 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 9738
I20240502 20:46:42.519460 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 9870
I20240502 20:46:42.525709 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 10002
I20240502 20:46:42.527195 9472 heartbeater.cc:503] Master 127.22.45.126:41377 was elected leader, sending a full tablet report...
I20240502 20:46:42.530256 22709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu with pid 10136
I20240502 20:46:42.687652 9603 heartbeater.cc:503] Master 127.22.45.126:41377 was elected leader, sending a full tablet report...
I20240502 20:46:42.867545 9735 heartbeater.cc:503] Master 127.22.45.126:41377 was elected leader, sending a full tablet report...
I20240502 20:46:46.285845 9145 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33288:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:46.305269 9669 tablet_service.cc:1467] Processing CreateTablet for tablet edebc185b24441da9504c78394378f4d (DEFAULT_TABLE table=rebalance_test_table_0 [id=ac354d7ce0fb41799e37a7e563898af0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:46.305269 9269 tablet_service.cc:1467] Processing CreateTablet for tablet 94eebae830c4437d8dc4149e7f1d8ef1 (DEFAULT_TABLE table=rebalance_test_table_0 [id=ac354d7ce0fb41799e37a7e563898af0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:46.305270 9402 tablet_service.cc:1467] Processing CreateTablet for tablet fe22224142984d69a2def286584b9aaa (DEFAULT_TABLE table=rebalance_test_table_0 [id=ac354d7ce0fb41799e37a7e563898af0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:46.305270 9404 tablet_service.cc:1467] Processing CreateTablet for tablet edebc185b24441da9504c78394378f4d (DEFAULT_TABLE table=rebalance_test_table_0 [id=ac354d7ce0fb41799e37a7e563898af0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:46.306195 9669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edebc185b24441da9504c78394378f4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:46.306195 9269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94eebae830c4437d8dc4149e7f1d8ef1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:46.306196 9404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edebc185b24441da9504c78394378f4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:46.306329 9403 tablet_service.cc:1467] Processing CreateTablet for tablet 94eebae830c4437d8dc4149e7f1d8ef1 (DEFAULT_TABLE table=rebalance_test_table_0 [id=ac354d7ce0fb41799e37a7e563898af0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:46.306560 9403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94eebae830c4437d8dc4149e7f1d8ef1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:46.307387 9268 tablet_service.cc:1467] Processing CreateTablet for tablet fe22224142984d69a2def286584b9aaa (DEFAULT_TABLE table=rebalance_test_table_0 [id=ac354d7ce0fb41799e37a7e563898af0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:46.307601 9268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe22224142984d69a2def286584b9aaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:46.307639 9402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe22224142984d69a2def286584b9aaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:46.308137 9668 tablet_service.cc:1467] Processing CreateTablet for tablet fe22224142984d69a2def286584b9aaa (DEFAULT_TABLE table=rebalance_test_table_0 [id=ac354d7ce0fb41799e37a7e563898af0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:46.309764 9668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe22224142984d69a2def286584b9aaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:46.313097 10287 tablet_bootstrap.cc:492] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f: Bootstrap starting.
I20240502 20:46:46.313480 10286 tablet_bootstrap.cc:492] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc: Bootstrap starting.
I20240502 20:46:46.315277 10287 tablet_bootstrap.cc:654] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:46.315292 10286 tablet_bootstrap.cc:654] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:46.315706 9537 tablet_service.cc:1467] Processing CreateTablet for tablet edebc185b24441da9504c78394378f4d (DEFAULT_TABLE table=rebalance_test_table_0 [id=ac354d7ce0fb41799e37a7e563898af0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:46.316226 10287 log.cc:824] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f: Log is configured to *not* fsync() on all Append() calls
I20240502 20:46:46.316576 9537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edebc185b24441da9504c78394378f4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:46.316702 10286 log.cc:824] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc: Log is configured to *not* fsync() on all Append() calls
I20240502 20:46:46.317468 9536 tablet_service.cc:1467] Processing CreateTablet for tablet 94eebae830c4437d8dc4149e7f1d8ef1 (DEFAULT_TABLE table=rebalance_test_table_0 [id=ac354d7ce0fb41799e37a7e563898af0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:46.317716 9536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94eebae830c4437d8dc4149e7f1d8ef1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:46.318209 10286 tablet_bootstrap.cc:492] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc: No bootstrap required, opened a new log
I20240502 20:46:46.318380 10286 ts_tablet_manager.cc:1367] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240502 20:46:46.318399 10287 tablet_bootstrap.cc:492] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f: No bootstrap required, opened a new log
I20240502 20:46:46.318526 10288 tablet_bootstrap.cc:492] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448: Bootstrap starting.
I20240502 20:46:46.318720 10287 ts_tablet_manager.cc:1367] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20240502 20:46:46.320232 10288 tablet_bootstrap.cc:654] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:46.320967 10288 log.cc:824] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448: Log is configured to *not* fsync() on all Append() calls
I20240502 20:46:46.322407 10288 tablet_bootstrap.cc:492] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448: No bootstrap required, opened a new log
I20240502 20:46:46.322432 10286 raft_consensus.cc:348] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:46.322593 10288 ts_tablet_manager.cc:1367] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20240502 20:46:46.322655 10286 raft_consensus.cc:374] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:46.322727 10286 raft_consensus.cc:729] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Initialized, Role: FOLLOWER
I20240502 20:46:46.322816 10287 raft_consensus.cc:348] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:46.323081 10287 raft_consensus.cc:374] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:46.323179 10287 raft_consensus.cc:729] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Initialized, Role: FOLLOWER
I20240502 20:46:46.323184 10286 consensus_queue.cc:260] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:46.323869 10287 consensus_queue.cc:260] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:46.324014 10286 ts_tablet_manager.cc:1398] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20240502 20:46:46.324188 10292 tablet_bootstrap.cc:492] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e: Bootstrap starting.
I20240502 20:46:46.324343 10286 tablet_bootstrap.cc:492] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc: Bootstrap starting.
I20240502 20:46:46.325359 10286 tablet_bootstrap.cc:654] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:46.325884 10292 tablet_bootstrap.cc:654] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:46.325942 10288 raft_consensus.cc:348] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:46.326103 10288 raft_consensus.cc:374] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:46.326153 10288 raft_consensus.cc:729] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Initialized, Role: FOLLOWER
I20240502 20:46:46.326443 10286 tablet_bootstrap.cc:492] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc: No bootstrap required, opened a new log
I20240502 20:46:46.326537 10286 ts_tablet_manager.cc:1367] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:46.326598 10292 log.cc:824] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e: Log is configured to *not* fsync() on all Append() calls
I20240502 20:46:46.326545 10288 consensus_queue.cc:260] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:46.327039 10286 raft_consensus.cc:348] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:46.327134 10286 raft_consensus.cc:374] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:46.327164 10286 raft_consensus.cc:729] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Initialized, Role: FOLLOWER
I20240502 20:46:46.327296 10288 ts_tablet_manager.cc:1398] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20240502 20:46:46.327302 10286 consensus_queue.cc:260] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:46.327623 10288 tablet_bootstrap.cc:492] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448: Bootstrap starting.
I20240502 20:46:46.327982 10292 tablet_bootstrap.cc:492] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e: No bootstrap required, opened a new log
I20240502 20:46:46.328133 10292 ts_tablet_manager.cc:1367] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 20:46:46.328258 10287 ts_tablet_manager.cc:1398] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20240502 20:46:46.328464 10288 tablet_bootstrap.cc:654] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:46.329468 10288 tablet_bootstrap.cc:492] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448: No bootstrap required, opened a new log
I20240502 20:46:46.329547 10288 ts_tablet_manager.cc:1367] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:46.329849 10286 ts_tablet_manager.cc:1398] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240502 20:46:46.330019 10286 tablet_bootstrap.cc:492] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc: Bootstrap starting.
I20240502 20:46:46.330022 10288 raft_consensus.cc:348] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:46.330109 10288 raft_consensus.cc:374] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:46.330145 10288 raft_consensus.cc:729] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Initialized, Role: FOLLOWER
I20240502 20:46:46.330271 10288 consensus_queue.cc:260] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:46.330648 10288 ts_tablet_manager.cc:1398] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 20:46:46.331210 10286 tablet_bootstrap.cc:654] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:46.331801 10287 tablet_bootstrap.cc:492] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f: Bootstrap starting.
I20240502 20:46:46.332352 10292 raft_consensus.cc:348] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:46.332540 10292 raft_consensus.cc:374] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:46.332594 10292 raft_consensus.cc:729] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Initialized, Role: FOLLOWER
I20240502 20:46:46.332784 10287 tablet_bootstrap.cc:654] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:46.332967 10292 consensus_queue.cc:260] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:46.333690 10292 ts_tablet_manager.cc:1398] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240502 20:46:46.333916 10292 tablet_bootstrap.cc:492] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e: Bootstrap starting.
I20240502 20:46:46.334604 10286 tablet_bootstrap.cc:492] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc: No bootstrap required, opened a new log
I20240502 20:46:46.334681 10286 ts_tablet_manager.cc:1367] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240502 20:46:46.334744 10287 tablet_bootstrap.cc:492] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f: No bootstrap required, opened a new log
I20240502 20:46:46.334817 10287 ts_tablet_manager.cc:1367] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:46.335069 10286 raft_consensus.cc:348] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:46.335151 10286 raft_consensus.cc:374] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:46.335180 10286 raft_consensus.cc:729] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Initialized, Role: FOLLOWER
I20240502 20:46:46.335278 10286 consensus_queue.cc:260] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:46.335290 10287 raft_consensus.cc:348] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:46.335372 10292 tablet_bootstrap.cc:654] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:46.335378 10287 raft_consensus.cc:374] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:46.335408 10287 raft_consensus.cc:729] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Initialized, Role: FOLLOWER
I20240502 20:46:46.335507 10286 ts_tablet_manager.cc:1398] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:46.335523 10287 consensus_queue.cc:260] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:46.335783 10287 ts_tablet_manager.cc:1398] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:46.336395 10292 tablet_bootstrap.cc:492] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e: No bootstrap required, opened a new log
I20240502 20:46:46.336478 10292 ts_tablet_manager.cc:1367] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:46.336953 10292 raft_consensus.cc:348] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:46.337041 10292 raft_consensus.cc:374] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:46.337074 10292 raft_consensus.cc:729] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Initialized, Role: FOLLOWER
I20240502 20:46:46.337185 10292 consensus_queue.cc:260] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:46.337432 10292 ts_tablet_manager.cc:1398] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20240502 20:46:46.359805 9736 tablet.cc:2369] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 20:46:46.362730 9338 tablet.cc:2369] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 20:46:46.396075 10298 raft_consensus.cc:482] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:46.396229 10298 raft_consensus.cc:504] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:46.397294 10298 leader_election.cc:288] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:46.400837 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" is_pre_election: true
I20240502 20:46:46.400986 9557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" is_pre_election: true
I20240502 20:46:46.401099 9289 raft_consensus.cc:2454] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 0.
I20240502 20:46:46.401229 9557 raft_consensus.cc:2454] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 0.
I20240502 20:46:46.401525 9355 leader_election.cc:302] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:46.401742 10298 raft_consensus.cc:2781] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:46.401813 10298 raft_consensus.cc:482] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:46.401858 10298 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:46.402678 10298 raft_consensus.cc:504] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:46.402957 10298 leader_election.cc:288] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:46.403264 9557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:46.403264 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:46.403374 9289 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:46.403374 9557 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:46.404321 9289 raft_consensus.cc:2454] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 1.
I20240502 20:46:46.404333 9557 raft_consensus.cc:2454] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 1.
I20240502 20:46:46.404595 9357 leader_election.cc:302] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:46.404744 10298 raft_consensus.cc:2781] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:46.404835 10298 raft_consensus.cc:686] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:46.405092 10298 consensus_queue.cc:237] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:46.407578 9144 catalog_manager.cc:5484] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 1 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
W20240502 20:46:46.428653 9604 tablet.cc:2369] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 20:46:46.519447 9473 tablet.cc:2369] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 20:46:46.654395 10302 raft_consensus.cc:482] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:46.654625 10302 raft_consensus.cc:504] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:46.655673 10302 leader_election.cc:288] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:46.659341 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe22224142984d69a2def286584b9aaa" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" is_pre_election: true
I20240502 20:46:46.659368 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe22224142984d69a2def286584b9aaa" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc" is_pre_election: true
I20240502 20:46:46.659471 9289 raft_consensus.cc:2454] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 0.
I20240502 20:46:46.659564 9424 raft_consensus.cc:2454] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 0.
I20240502 20:46:46.659827 9621 leader_election.cc:302] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:46.660072 10302 raft_consensus.cc:2781] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:46.660145 10302 raft_consensus.cc:482] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:46.660187 10302 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:46.660961 10302 raft_consensus.cc:504] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:46.661252 10302 leader_election.cc:288] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:46.661568 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe22224142984d69a2def286584b9aaa" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:46.661569 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe22224142984d69a2def286584b9aaa" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:46.661684 9289 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:46.661684 9424 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:46.662600 9424 raft_consensus.cc:2454] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 1.
I20240502 20:46:46.662601 9289 raft_consensus.cc:2454] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 1.
I20240502 20:46:46.662873 9622 leader_election.cc:302] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:46.663022 10302 raft_consensus.cc:2781] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:46.663116 10302 raft_consensus.cc:686] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 1 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:46.663388 10302 consensus_queue.cc:237] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:46.665949 9144 catalog_manager.cc:5484] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:46.707183 10302 raft_consensus.cc:482] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:46.707273 10302 raft_consensus.cc:504] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:46.707651 10302 leader_election.cc:288] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:46.707973 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edebc185b24441da9504c78394378f4d" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc" is_pre_election: true
I20240502 20:46:46.708118 9424 raft_consensus.cc:2454] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 0.
I20240502 20:46:46.708364 9622 leader_election.cc:302] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:46.708544 10302 raft_consensus.cc:2781] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:46.708595 10302 raft_consensus.cc:482] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:46.708618 10302 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:46.709333 10302 raft_consensus.cc:504] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:46.709621 10302 leader_election.cc:288] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:46.709915 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edebc185b24441da9504c78394378f4d" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:46.710003 9424 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:46.710880 9424 raft_consensus.cc:2454] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 1.
I20240502 20:46:46.710969 9557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edebc185b24441da9504c78394378f4d" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" is_pre_election: true
I20240502 20:46:46.711053 9556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edebc185b24441da9504c78394378f4d" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:46.711105 9557 raft_consensus.cc:2454] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 0.
I20240502 20:46:46.711124 9622 leader_election.cc:302] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:46.711277 10302 raft_consensus.cc:2781] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:46.711423 10302 raft_consensus.cc:686] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 1 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:46.711565 10302 consensus_queue.cc:237] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:46.712978 9144 catalog_manager.cc:5484] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:46.798395 10298 consensus_queue.cc:1035] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:46.803220 10298 consensus_queue.cc:1035] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 20:46:46.848238 10314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:46.848438 10314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:46.851366 10314 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20240502 20:46:46.932538 9556 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:46.933193 9424 raft_consensus.cc:1261] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:46.933728 9556 raft_consensus.cc:1261] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:46.934099 10302 consensus_queue.cc:1035] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:46.934588 10295 consensus_queue.cc:1035] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:46.938669 9423 raft_consensus.cc:1261] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:46.938771 9289 raft_consensus.cc:1261] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:46.939116 10295 consensus_queue.cc:1035] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:46.939420 10295 consensus_queue.cc:1035] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:46.953058 10316 mvcc.cc:205] Tried to move back new op lower bound from 7023340777189269504 to 7023340776291631104. Current Snapshot: MvccSnapshot[applied={T|T < 7023340777189269504}]
I20240502 20:46:46.953223 10347 mvcc.cc:205] Tried to move back new op lower bound from 7023340777189269504 to 7023340776291631104. Current Snapshot: MvccSnapshot[applied={T|T < 7023340777189269504}]
I20240502 20:46:46.961479 10317 mvcc.cc:205] Tried to move back new op lower bound from 7023340777216557056 to 7023340776095543296. Current Snapshot: MvccSnapshot[applied={T|T < 7023340777216557056}]
I20240502 20:46:46.984262 10355 mvcc.cc:205] Tried to move back new op lower bound from 7023340777189269504 to 7023340776291631104. Current Snapshot: MvccSnapshot[applied={T|T < 7023340777189269504}]
INSERT report
rows total: 2000
time total: 103.96 ms
time per row: 0.05198 ms
I20240502 20:46:46.997023 9144 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33288:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:47.001797 9537 tablet_service.cc:1467] Processing CreateTablet for tablet b6842838ea9b47d7bd7c343a16ee8768 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8f819e1b1f254a1ca57db59b8cd80da9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:47.002074 9537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6842838ea9b47d7bd7c343a16ee8768. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:47.002033 9269 tablet_service.cc:1467] Processing CreateTablet for tablet b6842838ea9b47d7bd7c343a16ee8768 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8f819e1b1f254a1ca57db59b8cd80da9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:47.002266 9269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6842838ea9b47d7bd7c343a16ee8768. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:47.002882 9668 tablet_service.cc:1467] Processing CreateTablet for tablet acd3076bc7344e7694362ea9ffa35e54 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8f819e1b1f254a1ca57db59b8cd80da9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:47.002877 9536 tablet_service.cc:1467] Processing CreateTablet for tablet acd3076bc7344e7694362ea9ffa35e54 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8f819e1b1f254a1ca57db59b8cd80da9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:47.003115 9668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acd3076bc7344e7694362ea9ffa35e54. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:47.003114 9536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acd3076bc7344e7694362ea9ffa35e54. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:47.004462 10368 tablet_bootstrap.cc:492] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f: Bootstrap starting.
I20240502 20:46:47.004451 9535 tablet_service.cc:1467] Processing CreateTablet for tablet 3315525f1849462fbf49a285f424ea51 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8f819e1b1f254a1ca57db59b8cd80da9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:47.004668 9535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3315525f1849462fbf49a285f424ea51. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:47.005326 10368 tablet_bootstrap.cc:654] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:47.005661 9403 tablet_service.cc:1467] Processing CreateTablet for tablet 3315525f1849462fbf49a285f424ea51 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8f819e1b1f254a1ca57db59b8cd80da9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:47.005869 10369 tablet_bootstrap.cc:492] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448: Bootstrap starting.
I20240502 20:46:47.005913 9403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3315525f1849462fbf49a285f424ea51. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:47.006008 10367 tablet_bootstrap.cc:492] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e: Bootstrap starting.
I20240502 20:46:47.006618 10369 tablet_bootstrap.cc:654] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:47.006661 9669 tablet_service.cc:1467] Processing CreateTablet for tablet b6842838ea9b47d7bd7c343a16ee8768 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8f819e1b1f254a1ca57db59b8cd80da9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:47.006872 9669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6842838ea9b47d7bd7c343a16ee8768. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:47.006925 10367 tablet_bootstrap.cc:654] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:47.007963 10367 tablet_bootstrap.cc:492] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e: No bootstrap required, opened a new log
I20240502 20:46:47.008033 10367 ts_tablet_manager.cc:1367] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:47.008265 10374 tablet_bootstrap.cc:492] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc: Bootstrap starting.
I20240502 20:46:47.008483 10367 raft_consensus.cc:348] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:47.008595 10367 raft_consensus.cc:374] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:47.008628 10367 raft_consensus.cc:729] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Initialized, Role: FOLLOWER
I20240502 20:46:47.008745 10367 consensus_queue.cc:260] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:47.009001 10367 ts_tablet_manager.cc:1398] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:47.009191 10367 tablet_bootstrap.cc:492] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e: Bootstrap starting.
I20240502 20:46:47.009429 10369 tablet_bootstrap.cc:492] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448: No bootstrap required, opened a new log
I20240502 20:46:47.009500 10369 ts_tablet_manager.cc:1367] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:47.009469 9268 tablet_service.cc:1467] Processing CreateTablet for tablet 3315525f1849462fbf49a285f424ea51 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8f819e1b1f254a1ca57db59b8cd80da9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:47.009672 9268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3315525f1849462fbf49a285f424ea51. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:47.009696 10368 tablet_bootstrap.cc:492] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f: No bootstrap required, opened a new log
I20240502 20:46:47.009759 10368 ts_tablet_manager.cc:1367] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 20:46:47.009889 10369 raft_consensus.cc:348] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:47.009994 10369 raft_consensus.cc:374] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:47.010023 10367 tablet_bootstrap.cc:654] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:47.010023 10369 raft_consensus.cc:729] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Initialized, Role: FOLLOWER
I20240502 20:46:47.010150 10368 raft_consensus.cc:348] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:47.010248 10368 raft_consensus.cc:374] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:47.010277 10368 raft_consensus.cc:729] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Initialized, Role: FOLLOWER
I20240502 20:46:47.010411 10369 consensus_queue.cc:260] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:47.010391 10368 consensus_queue.cc:260] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:47.010728 10368 ts_tablet_manager.cc:1398] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:47.011037 10374 tablet_bootstrap.cc:654] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:47.011878 9269 tablet_service.cc:1467] Processing CreateTablet for tablet acd3076bc7344e7694362ea9ffa35e54 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8f819e1b1f254a1ca57db59b8cd80da9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:47.012123 9269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acd3076bc7344e7694362ea9ffa35e54. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:47.012140 10369 ts_tablet_manager.cc:1398] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:47.012373 10369 tablet_bootstrap.cc:492] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448: Bootstrap starting.
I20240502 20:46:47.013612 10369 tablet_bootstrap.cc:654] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:47.014175 10368 tablet_bootstrap.cc:492] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f: Bootstrap starting.
I20240502 20:46:47.015079 10368 tablet_bootstrap.cc:654] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:47.015285 10369 tablet_bootstrap.cc:492] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448: No bootstrap required, opened a new log
I20240502 20:46:47.015369 10369 ts_tablet_manager.cc:1367] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:47.015779 10369 raft_consensus.cc:348] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:47.015870 10369 raft_consensus.cc:374] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:47.015898 10369 raft_consensus.cc:729] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Initialized, Role: FOLLOWER
I20240502 20:46:47.016002 10369 consensus_queue.cc:260] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:47.016245 10369 ts_tablet_manager.cc:1398] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:47.016268 10368 tablet_bootstrap.cc:492] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f: No bootstrap required, opened a new log
I20240502 20:46:47.016335 10368 ts_tablet_manager.cc:1367] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:47.016719 10368 raft_consensus.cc:348] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.016800 10368 raft_consensus.cc:374] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:47.016829 10368 raft_consensus.cc:729] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Initialized, Role: FOLLOWER
I20240502 20:46:47.016934 10368 consensus_queue.cc:260] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.017154 10368 ts_tablet_manager.cc:1398] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:47.017304 10368 tablet_bootstrap.cc:492] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f: Bootstrap starting.
I20240502 20:46:47.018095 10368 tablet_bootstrap.cc:654] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:47.019155 10367 tablet_bootstrap.cc:492] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e: No bootstrap required, opened a new log
I20240502 20:46:47.019243 10367 ts_tablet_manager.cc:1367] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20240502 20:46:47.019506 10374 tablet_bootstrap.cc:492] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc: No bootstrap required, opened a new log
I20240502 20:46:47.019631 10374 ts_tablet_manager.cc:1367] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 20:46:47.019662 10367 raft_consensus.cc:348] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:47.019757 10367 raft_consensus.cc:374] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:47.019795 10367 raft_consensus.cc:729] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Initialized, Role: FOLLOWER
I20240502 20:46:47.019906 10367 consensus_queue.cc:260] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:47.019945 10368 tablet_bootstrap.cc:492] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f: No bootstrap required, opened a new log
I20240502 20:46:47.020011 10368 ts_tablet_manager.cc:1367] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:47.020146 10367 ts_tablet_manager.cc:1398] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:47.020319 10367 tablet_bootstrap.cc:492] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e: Bootstrap starting.
I20240502 20:46:47.020396 10368 raft_consensus.cc:348] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:47.020490 10368 raft_consensus.cc:374] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:47.020519 10368 raft_consensus.cc:729] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Initialized, Role: FOLLOWER
I20240502 20:46:47.020615 10368 consensus_queue.cc:260] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:47.020829 10368 ts_tablet_manager.cc:1398] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:47.020848 10374 raft_consensus.cc:348] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.020959 10374 raft_consensus.cc:374] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:47.020990 10374 raft_consensus.cc:729] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Initialized, Role: FOLLOWER
I20240502 20:46:47.021126 10374 consensus_queue.cc:260] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.021333 10367 tablet_bootstrap.cc:654] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:47.022521 10374 ts_tablet_manager.cc:1398] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:47.022539 10367 tablet_bootstrap.cc:492] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e: No bootstrap required, opened a new log
I20240502 20:46:47.022914 10367 ts_tablet_manager.cc:1367] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 20:46:47.023326 10367 raft_consensus.cc:348] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.023424 10367 raft_consensus.cc:374] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:47.023458 10367 raft_consensus.cc:729] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Initialized, Role: FOLLOWER
I20240502 20:46:47.023567 10367 consensus_queue.cc:260] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.023795 10367 ts_tablet_manager.cc:1398] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:47.069187 10304 raft_consensus.cc:482] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:47.069290 10304 raft_consensus.cc:504] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:47.070358 10304 leader_election.cc:288] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:47.073700 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" is_pre_election: true
I20240502 20:46:47.073807 9689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64136bd82a30486e9d56fc28da4bd448" is_pre_election: true
I20240502 20:46:47.073850 9289 raft_consensus.cc:2454] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 0.
I20240502 20:46:47.074002 9689 raft_consensus.cc:2454] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 0.
I20240502 20:46:47.074185 9490 leader_election.cc:302] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:47.074398 10304 raft_consensus.cc:2781] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:47.074476 10304 raft_consensus.cc:482] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:47.074523 10304 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:47.075284 10304 raft_consensus.cc:504] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:47.075585 10304 leader_election.cc:288] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 1 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:47.075891 9689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:47.076004 9689 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:47.076364 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:47.076481 9289 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:47.076948 9689 raft_consensus.cc:2454] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 1.
I20240502 20:46:47.077219 9492 leader_election.cc:302] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:47.077392 10304 raft_consensus.cc:2781] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:47.077432 9289 raft_consensus.cc:2454] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 1.
I20240502 20:46:47.077593 10304 raft_consensus.cc:686] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:47.077849 10304 consensus_queue.cc:237] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:47.079505 9144 catalog_manager.cc:5484] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 1 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:47.218602 10300 raft_consensus.cc:482] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:47.218737 10300 raft_consensus.cc:504] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.219739 10300 leader_election.cc:288] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:47.223088 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3315525f1849462fbf49a285f424ea51" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc" is_pre_election: true
I20240502 20:46:47.223169 9557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3315525f1849462fbf49a285f424ea51" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" is_pre_election: true
I20240502 20:46:47.223219 9424 raft_consensus.cc:2454] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 0.
I20240502 20:46:47.223318 9557 raft_consensus.cc:2454] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 0.
I20240502 20:46:47.223554 9222 leader_election.cc:302] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:47.223773 10300 raft_consensus.cc:2781] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:47.223831 10300 raft_consensus.cc:482] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:47.223865 10300 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:47.224632 10300 raft_consensus.cc:504] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.224926 10300 leader_election.cc:288] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 1 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:47.225240 9557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3315525f1849462fbf49a285f424ea51" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:47.225240 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3315525f1849462fbf49a285f424ea51" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:47.225358 9424 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:47.225358 9557 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:47.226225 9557 raft_consensus.cc:2454] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 1.
I20240502 20:46:47.226226 9424 raft_consensus.cc:2454] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 1.
I20240502 20:46:47.226528 9223 leader_election.cc:302] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:47.226683 10300 raft_consensus.cc:2781] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:47.226882 10300 raft_consensus.cc:686] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:47.227164 10300 consensus_queue.cc:237] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.228816 9144 catalog_manager.cc:5484] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 1 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:47.271212 10295 raft_consensus.cc:482] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:47.271323 10295 raft_consensus.cc:504] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:47.271682 10295 leader_election.cc:288] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:47.272039 9557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" is_pre_election: true
I20240502 20:46:47.272049 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" is_pre_election: true
I20240502 20:46:47.272166 9557 raft_consensus.cc:2454] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 0.
I20240502 20:46:47.272177 9289 raft_consensus.cc:2454] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 0.
I20240502 20:46:47.272455 9623 leader_election.cc:302] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:47.272617 10295 raft_consensus.cc:2781] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:47.272676 10295 raft_consensus.cc:482] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:47.272711 10295 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:47.273430 10295 raft_consensus.cc:504] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:47.273738 10295 leader_election.cc:288] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:47.274037 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:47.274051 9557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:47.274125 9289 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:47.274137 9557 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:47.274827 9289 raft_consensus.cc:2454] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 1.
I20240502 20:46:47.274875 9557 raft_consensus.cc:2454] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 1.
I20240502 20:46:47.275087 9621 leader_election.cc:302] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:47.275271 10295 raft_consensus.cc:2781] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:47.275359 10295 raft_consensus.cc:686] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 1 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:47.275489 10295 consensus_queue.cc:237] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:47.276902 9144 catalog_manager.cc:5484] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
W20240502 20:46:47.394852 10392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:47.395097 10392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:47.398079 10392 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20240502 20:46:47.470265 9557 raft_consensus.cc:1261] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:47.470268 9424 raft_consensus.cc:1261] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:47.470759 10300 consensus_queue.cc:1035] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:47.471290 10388 consensus_queue.cc:1035] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:47.474828 9689 raft_consensus.cc:1261] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:47.475448 10383 consensus_queue.cc:1035] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:47.476259 9289 raft_consensus.cc:1261] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:47.476621 10383 consensus_queue.cc:1035] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:47.479082 9556 raft_consensus.cc:1261] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:47.479449 10295 consensus_queue.cc:1035] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:47.482095 9289 raft_consensus.cc:1261] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:47.483525 10295 consensus_queue.cc:1035] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 103.262 ms
time per row: 0.051631 ms
I20240502 20:46:47.541594 9143 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33288:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:47.546591 9403 tablet_service.cc:1467] Processing CreateTablet for tablet d92bfee8fabd46a28ffcb7e319879406 (DEFAULT_TABLE table=rebalance_test_table_2 [id=4f819aa660774908a9dce3de521c6de6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:47.546878 9403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d92bfee8fabd46a28ffcb7e319879406. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:47.546865 9269 tablet_service.cc:1467] Processing CreateTablet for tablet d92bfee8fabd46a28ffcb7e319879406 (DEFAULT_TABLE table=rebalance_test_table_2 [id=4f819aa660774908a9dce3de521c6de6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:47.547138 9269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d92bfee8fabd46a28ffcb7e319879406. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:47.547597 9268 tablet_service.cc:1467] Processing CreateTablet for tablet cb8d94e044dd4480a7948bc3d4aca7e2 (DEFAULT_TABLE table=rebalance_test_table_2 [id=4f819aa660774908a9dce3de521c6de6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:47.547809 9268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb8d94e044dd4480a7948bc3d4aca7e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:47.547871 9535 tablet_service.cc:1467] Processing CreateTablet for tablet e553edf81f684b11aaafa549b5f8c7b7 (DEFAULT_TABLE table=rebalance_test_table_2 [id=4f819aa660774908a9dce3de521c6de6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:47.548101 9535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e553edf81f684b11aaafa549b5f8c7b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:47.549161 9669 tablet_service.cc:1467] Processing CreateTablet for tablet d92bfee8fabd46a28ffcb7e319879406 (DEFAULT_TABLE table=rebalance_test_table_2 [id=4f819aa660774908a9dce3de521c6de6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:47.549394 9669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d92bfee8fabd46a28ffcb7e319879406. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:47.550277 9402 tablet_service.cc:1467] Processing CreateTablet for tablet e553edf81f684b11aaafa549b5f8c7b7 (DEFAULT_TABLE table=rebalance_test_table_2 [id=4f819aa660774908a9dce3de521c6de6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:47.550277 9668 tablet_service.cc:1467] Processing CreateTablet for tablet e553edf81f684b11aaafa549b5f8c7b7 (DEFAULT_TABLE table=rebalance_test_table_2 [id=4f819aa660774908a9dce3de521c6de6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:47.550518 9668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e553edf81f684b11aaafa549b5f8c7b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:47.550534 9402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e553edf81f684b11aaafa549b5f8c7b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:47.550967 9404 tablet_service.cc:1467] Processing CreateTablet for tablet cb8d94e044dd4480a7948bc3d4aca7e2 (DEFAULT_TABLE table=rebalance_test_table_2 [id=4f819aa660774908a9dce3de521c6de6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:47.551157 9667 tablet_service.cc:1467] Processing CreateTablet for tablet cb8d94e044dd4480a7948bc3d4aca7e2 (DEFAULT_TABLE table=rebalance_test_table_2 [id=4f819aa660774908a9dce3de521c6de6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:47.551379 9667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb8d94e044dd4480a7948bc3d4aca7e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:47.552507 9404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb8d94e044dd4480a7948bc3d4aca7e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:47.552868 10447 tablet_bootstrap.cc:492] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e: Bootstrap starting.
I20240502 20:46:47.553791 10448 tablet_bootstrap.cc:492] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc: Bootstrap starting.
I20240502 20:46:47.553982 10447 tablet_bootstrap.cc:654] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:47.554636 10448 tablet_bootstrap.cc:654] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:47.554922 10446 tablet_bootstrap.cc:492] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f: Bootstrap starting.
I20240502 20:46:47.555259 10450 tablet_bootstrap.cc:492] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448: Bootstrap starting.
I20240502 20:46:47.555418 10447 tablet_bootstrap.cc:492] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e: No bootstrap required, opened a new log
I20240502 20:46:47.555552 10447 ts_tablet_manager.cc:1367] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 20:46:47.555601 10448 tablet_bootstrap.cc:492] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc: No bootstrap required, opened a new log
I20240502 20:46:47.555713 10446 tablet_bootstrap.cc:654] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:47.555733 10448 ts_tablet_manager.cc:1367] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:47.556061 10450 tablet_bootstrap.cc:654] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:47.556028 10447 raft_consensus.cc:348] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:47.556126 10447 raft_consensus.cc:374] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:47.556106 10448 raft_consensus.cc:348] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.556187 10448 raft_consensus.cc:374] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:47.556236 10448 raft_consensus.cc:729] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Initialized, Role: FOLLOWER
I20240502 20:46:47.556270 10447 raft_consensus.cc:729] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Initialized, Role: FOLLOWER
I20240502 20:46:47.556406 10448 consensus_queue.cc:260] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.556463 10447 consensus_queue.cc:260] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:47.556788 10448 ts_tablet_manager.cc:1398] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:47.556980 10448 tablet_bootstrap.cc:492] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc: Bootstrap starting.
I20240502 20:46:47.557070 10446 tablet_bootstrap.cc:492] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f: No bootstrap required, opened a new log
I20240502 20:46:47.557158 10446 ts_tablet_manager.cc:1367] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:47.557582 10446 raft_consensus.cc:348] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.557688 10446 raft_consensus.cc:374] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:47.557725 10446 raft_consensus.cc:729] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Initialized, Role: FOLLOWER
I20240502 20:46:47.557739 10450 tablet_bootstrap.cc:492] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448: No bootstrap required, opened a new log
I20240502 20:46:47.557807 10450 ts_tablet_manager.cc:1367] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:47.557859 10446 consensus_queue.cc:260] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.558113 10446 ts_tablet_manager.cc:1398] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:47.558213 10450 raft_consensus.cc:348] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.558280 10446 tablet_bootstrap.cc:492] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f: Bootstrap starting.
I20240502 20:46:47.558322 10450 raft_consensus.cc:374] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:47.558364 10450 raft_consensus.cc:729] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Initialized, Role: FOLLOWER
I20240502 20:46:47.558502 10450 consensus_queue.cc:260] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.558761 10450 ts_tablet_manager.cc:1398] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:47.558910 10450 tablet_bootstrap.cc:492] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448: Bootstrap starting.
I20240502 20:46:47.559092 10446 tablet_bootstrap.cc:654] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:47.559440 10448 tablet_bootstrap.cc:654] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:47.559696 10450 tablet_bootstrap.cc:654] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:47.560281 10447 ts_tablet_manager.cc:1398] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent starting tablet: real 0.005s user 0.000s sys 0.001s
I20240502 20:46:47.560885 10450 tablet_bootstrap.cc:492] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448: No bootstrap required, opened a new log
I20240502 20:46:47.560973 10450 ts_tablet_manager.cc:1367] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:47.561383 10450 raft_consensus.cc:348] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:47.561482 10450 raft_consensus.cc:374] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:47.561515 10450 raft_consensus.cc:729] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Initialized, Role: FOLLOWER
I20240502 20:46:47.561640 10450 consensus_queue.cc:260] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:47.561698 10446 tablet_bootstrap.cc:492] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f: No bootstrap required, opened a new log
I20240502 20:46:47.561770 10446 ts_tablet_manager.cc:1367] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:47.561964 10450 ts_tablet_manager.cc:1398] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:47.562115 10450 tablet_bootstrap.cc:492] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448: Bootstrap starting.
I20240502 20:46:47.562117 10446 raft_consensus.cc:348] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:47.562182 10446 raft_consensus.cc:374] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:47.562201 10446 raft_consensus.cc:729] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Initialized, Role: FOLLOWER
I20240502 20:46:47.562289 10446 consensus_queue.cc:260] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:47.562559 10446 ts_tablet_manager.cc:1398] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:47.562942 10450 tablet_bootstrap.cc:654] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:47.563185 10448 tablet_bootstrap.cc:492] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc: No bootstrap required, opened a new log
I20240502 20:46:47.563457 10448 ts_tablet_manager.cc:1367] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20240502 20:46:47.563881 10448 raft_consensus.cc:348] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:47.563987 10448 raft_consensus.cc:374] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:47.564018 10448 raft_consensus.cc:729] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Initialized, Role: FOLLOWER
I20240502 20:46:47.564124 10448 consensus_queue.cc:260] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:47.564376 10448 ts_tablet_manager.cc:1398] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:47.564596 10448 tablet_bootstrap.cc:492] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc: Bootstrap starting.
I20240502 20:46:47.565065 10450 tablet_bootstrap.cc:492] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448: No bootstrap required, opened a new log
I20240502 20:46:47.565140 10450 ts_tablet_manager.cc:1367] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 20:46:47.565393 10448 tablet_bootstrap.cc:654] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:47.565466 10450 raft_consensus.cc:348] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:47.565567 10450 raft_consensus.cc:374] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:47.565598 10450 raft_consensus.cc:729] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Initialized, Role: FOLLOWER
I20240502 20:46:47.565723 10450 consensus_queue.cc:260] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:47.566308 10448 tablet_bootstrap.cc:492] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc: No bootstrap required, opened a new log
I20240502 20:46:47.566345 10450 ts_tablet_manager.cc:1398] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:47.566383 10448 ts_tablet_manager.cc:1367] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 20:46:47.566802 10448 raft_consensus.cc:348] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:47.566937 10448 raft_consensus.cc:374] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:47.566974 10448 raft_consensus.cc:729] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Initialized, Role: FOLLOWER
I20240502 20:46:47.567067 10448 consensus_queue.cc:260] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:47.567297 10448 ts_tablet_manager.cc:1398] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:47.576910 10295 raft_consensus.cc:482] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:47.576989 10295 raft_consensus.cc:504] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:47.577296 10295 leader_election.cc:288] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:47.577603 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc" is_pre_election: true
I20240502 20:46:47.577605 9288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" is_pre_election: true
I20240502 20:46:47.577704 9288 raft_consensus.cc:2454] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 0.
I20240502 20:46:47.577710 9424 raft_consensus.cc:2454] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 0.
I20240502 20:46:47.577977 9621 leader_election.cc:302] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:47.578140 10295 raft_consensus.cc:2781] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:47.578197 10295 raft_consensus.cc:482] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:47.578233 10295 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:47.579020 10295 raft_consensus.cc:504] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:47.579291 10295 leader_election.cc:288] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:47.579553 9288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:47.579566 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:47.579667 9288 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:47.579675 9424 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:47.580317 9424 raft_consensus.cc:2454] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 1.
I20240502 20:46:47.580399 9288 raft_consensus.cc:2454] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 1.
I20240502 20:46:47.580579 9622 leader_election.cc:302] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:47.580768 10295 raft_consensus.cc:2781] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:47.580837 10295 raft_consensus.cc:686] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 1 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:47.580950 10295 consensus_queue.cc:237] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:47.582208 9143 catalog_manager.cc:5484] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:47.650823 10293 raft_consensus.cc:482] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:47.650957 10293 raft_consensus.cc:504] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.651427 10293 leader_election.cc:288] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:47.651856 9288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d92bfee8fabd46a28ffcb7e319879406" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" is_pre_election: true
I20240502 20:46:47.652014 9288 raft_consensus.cc:2454] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 0.
I20240502 20:46:47.652307 9355 leader_election.cc:302] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:47.652477 10293 raft_consensus.cc:2781] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:47.652545 10293 raft_consensus.cc:482] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:47.652602 10293 raft_consensus.cc:3037] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:47.653409 10293 raft_consensus.cc:504] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.653710 10293 leader_election.cc:288] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:47.654105 9288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d92bfee8fabd46a28ffcb7e319879406" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:47.654217 9288 raft_consensus.cc:3037] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:47.655071 9288 raft_consensus.cc:2454] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 1.
I20240502 20:46:47.655335 9355 leader_election.cc:302] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:47.655500 10293 raft_consensus.cc:2781] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:47.655519 9689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d92bfee8fabd46a28ffcb7e319879406" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64136bd82a30486e9d56fc28da4bd448" is_pre_election: true
I20240502 20:46:47.655670 10293 raft_consensus.cc:686] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:47.655645 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d92bfee8fabd46a28ffcb7e319879406" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:47.655655 9689 raft_consensus.cc:2454] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 0.
I20240502 20:46:47.655810 10293 consensus_queue.cc:237] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.657277 9143 catalog_manager.cc:5484] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 1 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:47.711232 10295 raft_consensus.cc:482] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:47.711351 10295 raft_consensus.cc:504] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:47.711728 10295 leader_election.cc:288] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:47.712110 9556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" is_pre_election: true
I20240502 20:46:47.712111 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc" is_pre_election: true
I20240502 20:46:47.712234 9556 raft_consensus.cc:2454] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 0.
I20240502 20:46:47.712246 9424 raft_consensus.cc:2454] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 0.
I20240502 20:46:47.712539 9623 leader_election.cc:302] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:47.712698 10295 raft_consensus.cc:2781] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:47.712757 10295 raft_consensus.cc:482] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:47.712781 10295 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:47.713500 10295 raft_consensus.cc:504] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:47.713788 10295 leader_election.cc:288] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:47.714078 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:47.714114 9556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:47.714175 9424 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:47.714229 9556 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:47.714910 9424 raft_consensus.cc:2454] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 1.
I20240502 20:46:47.715037 9556 raft_consensus.cc:2454] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 1.
I20240502 20:46:47.715163 9622 leader_election.cc:302] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:47.715313 10295 raft_consensus.cc:2781] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:47.715399 10295 raft_consensus.cc:686] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 1 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:47.715538 10295 consensus_queue.cc:237] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:47.717056 9143 catalog_manager.cc:5484] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
W20240502 20:46:47.838235 10465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:47.838433 10465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:47.841411 10465 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20240502 20:46:47.911300 9424 raft_consensus.cc:1261] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:47.911545 9556 raft_consensus.cc:1261] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:47.911581 9423 raft_consensus.cc:1261] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:47.912091 9288 raft_consensus.cc:1261] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:47.912393 10491 consensus_queue.cc:1035] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:47.912665 10491 consensus_queue.cc:1035] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:47.912891 10491 consensus_queue.cc:1035] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:47.913112 10491 consensus_queue.cc:1035] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:47.917260 9289 raft_consensus.cc:1261] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:47.917757 9688 raft_consensus.cc:3037] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:47.918721 10462 consensus_queue.cc:1035] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:47.920523 9688 raft_consensus.cc:1261] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:47.921538 10462 consensus_queue.cc:1035] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 93.6642 ms
time per row: 0.0468321 ms
I20240502 20:46:47.972690 9138 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33288:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:47.977671 9404 tablet_service.cc:1467] Processing CreateTablet for tablet 546ac55ccb0b4e91a17074a8ee8e99ef (DEFAULT_TABLE table=rebalance_test_table_3 [id=6443b6a05af74b3ba3f2865748727f2b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:47.977671 9535 tablet_service.cc:1467] Processing CreateTablet for tablet 546ac55ccb0b4e91a17074a8ee8e99ef (DEFAULT_TABLE table=rebalance_test_table_3 [id=6443b6a05af74b3ba3f2865748727f2b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:47.977944 9404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 546ac55ccb0b4e91a17074a8ee8e99ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:47.977957 9535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 546ac55ccb0b4e91a17074a8ee8e99ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:47.978571 9269 tablet_service.cc:1467] Processing CreateTablet for tablet fddd0fdac3ae414a8b2610752e96864b (DEFAULT_TABLE table=rebalance_test_table_3 [id=6443b6a05af74b3ba3f2865748727f2b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:47.978803 9269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fddd0fdac3ae414a8b2610752e96864b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:47.980007 10448 tablet_bootstrap.cc:492] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc: Bootstrap starting.
I20240502 20:46:47.980177 9268 tablet_service.cc:1467] Processing CreateTablet for tablet 546ac55ccb0b4e91a17074a8ee8e99ef (DEFAULT_TABLE table=rebalance_test_table_3 [id=6443b6a05af74b3ba3f2865748727f2b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:47.980393 9268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 546ac55ccb0b4e91a17074a8ee8e99ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:47.980648 10446 tablet_bootstrap.cc:492] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f: Bootstrap starting.
I20240502 20:46:47.980841 10448 tablet_bootstrap.cc:654] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:47.981359 10447 tablet_bootstrap.cc:492] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e: Bootstrap starting.
I20240502 20:46:47.981403 10446 tablet_bootstrap.cc:654] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:47.981787 10448 tablet_bootstrap.cc:492] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc: No bootstrap required, opened a new log
I20240502 20:46:47.981859 10448 ts_tablet_manager.cc:1367] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:47.982169 10447 tablet_bootstrap.cc:654] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:47.982273 10448 raft_consensus.cc:348] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.982371 10448 raft_consensus.cc:374] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:47.982401 10448 raft_consensus.cc:729] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Initialized, Role: FOLLOWER
I20240502 20:46:47.982523 10448 consensus_queue.cc:260] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.982569 9667 tablet_service.cc:1467] Processing CreateTablet for tablet 61b6c9892e6c49b2a424aa06cdf3dfd9 (DEFAULT_TABLE table=rebalance_test_table_3 [id=6443b6a05af74b3ba3f2865748727f2b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:47.982684 9536 tablet_service.cc:1467] Processing CreateTablet for tablet fddd0fdac3ae414a8b2610752e96864b (DEFAULT_TABLE table=rebalance_test_table_3 [id=6443b6a05af74b3ba3f2865748727f2b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:47.982803 10448 ts_tablet_manager.cc:1398] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:47.982827 9667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61b6c9892e6c49b2a424aa06cdf3dfd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:47.982897 9536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fddd0fdac3ae414a8b2610752e96864b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:47.984484 9668 tablet_service.cc:1467] Processing CreateTablet for tablet fddd0fdac3ae414a8b2610752e96864b (DEFAULT_TABLE table=rebalance_test_table_3 [id=6443b6a05af74b3ba3f2865748727f2b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:47.984688 9668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fddd0fdac3ae414a8b2610752e96864b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:47.984807 9535 tablet_service.cc:1467] Processing CreateTablet for tablet 61b6c9892e6c49b2a424aa06cdf3dfd9 (DEFAULT_TABLE table=rebalance_test_table_3 [id=6443b6a05af74b3ba3f2865748727f2b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:47.985018 9535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61b6c9892e6c49b2a424aa06cdf3dfd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:47.986013 10450 tablet_bootstrap.cc:492] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448: Bootstrap starting.
I20240502 20:46:47.986742 10450 tablet_bootstrap.cc:654] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:47.986790 9404 tablet_service.cc:1467] Processing CreateTablet for tablet 61b6c9892e6c49b2a424aa06cdf3dfd9 (DEFAULT_TABLE table=rebalance_test_table_3 [id=6443b6a05af74b3ba3f2865748727f2b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:47.987010 9404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61b6c9892e6c49b2a424aa06cdf3dfd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:47.987692 10450 tablet_bootstrap.cc:492] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448: No bootstrap required, opened a new log
I20240502 20:46:47.987771 10450 ts_tablet_manager.cc:1367] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:47.987910 10446 tablet_bootstrap.cc:492] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f: No bootstrap required, opened a new log
I20240502 20:46:47.987982 10446 ts_tablet_manager.cc:1367] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 20:46:47.988026 10447 tablet_bootstrap.cc:492] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e: No bootstrap required, opened a new log
I20240502 20:46:47.988090 10447 ts_tablet_manager.cc:1367] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 20:46:47.988178 10450 raft_consensus.cc:348] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:47.988263 10450 raft_consensus.cc:374] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:47.988291 10450 raft_consensus.cc:729] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Initialized, Role: FOLLOWER
I20240502 20:46:47.988351 10446 raft_consensus.cc:348] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.988399 10450 consensus_queue.cc:260] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:47.988446 10446 raft_consensus.cc:374] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:47.988479 10446 raft_consensus.cc:729] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Initialized, Role: FOLLOWER
I20240502 20:46:47.988466 10447 raft_consensus.cc:348] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.988544 10447 raft_consensus.cc:374] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:47.988572 10447 raft_consensus.cc:729] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Initialized, Role: FOLLOWER
I20240502 20:46:47.988600 10446 consensus_queue.cc:260] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.988637 10450 ts_tablet_manager.cc:1398] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:47.988670 10447 consensus_queue.cc:260] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.988785 10450 tablet_bootstrap.cc:492] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448: Bootstrap starting.
I20240502 20:46:47.988855 10446 ts_tablet_manager.cc:1398] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:47.988912 10447 ts_tablet_manager.cc:1398] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:47.989012 10446 tablet_bootstrap.cc:492] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f: Bootstrap starting.
I20240502 20:46:47.989046 10447 tablet_bootstrap.cc:492] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e: Bootstrap starting.
I20240502 20:46:47.989317 10448 tablet_bootstrap.cc:492] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc: Bootstrap starting.
I20240502 20:46:47.989552 10450 tablet_bootstrap.cc:654] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:47.989785 10447 tablet_bootstrap.cc:654] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:47.989785 10446 tablet_bootstrap.cc:654] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:47.990027 10448 tablet_bootstrap.cc:654] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:47.991003 10448 tablet_bootstrap.cc:492] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc: No bootstrap required, opened a new log
I20240502 20:46:47.991127 10448 ts_tablet_manager.cc:1367] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:47.991588 10448 raft_consensus.cc:348] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:47.991740 10448 raft_consensus.cc:374] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:47.991820 10448 raft_consensus.cc:729] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Initialized, Role: FOLLOWER
I20240502 20:46:47.992019 10448 consensus_queue.cc:260] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:47.992126 10450 tablet_bootstrap.cc:492] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448: No bootstrap required, opened a new log
I20240502 20:46:47.992208 10450 ts_tablet_manager.cc:1367] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:47.992487 10448 ts_tablet_manager.cc:1398] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:47.992633 10450 raft_consensus.cc:348] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.992733 10450 raft_consensus.cc:374] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:47.992777 10450 raft_consensus.cc:729] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Initialized, Role: FOLLOWER
I20240502 20:46:47.992880 10450 consensus_queue.cc:260] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.993127 10450 ts_tablet_manager.cc:1398] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:47.994616 10447 tablet_bootstrap.cc:492] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e: No bootstrap required, opened a new log
I20240502 20:46:47.994616 10446 tablet_bootstrap.cc:492] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f: No bootstrap required, opened a new log
I20240502 20:46:47.994690 10447 ts_tablet_manager.cc:1367] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20240502 20:46:47.994691 10446 ts_tablet_manager.cc:1367] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 20:46:47.995057 10447 raft_consensus.cc:348] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.995075 10446 raft_consensus.cc:348] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.995143 10447 raft_consensus.cc:374] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:47.995148 10446 raft_consensus.cc:374] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:47.995172 10447 raft_consensus.cc:729] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Initialized, Role: FOLLOWER
I20240502 20:46:47.995177 10446 raft_consensus.cc:729] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Initialized, Role: FOLLOWER
I20240502 20:46:47.995273 10447 consensus_queue.cc:260] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.995273 10446 consensus_queue.cc:260] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:47.995514 10446 ts_tablet_manager.cc:1398] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:47.995514 10447 ts_tablet_manager.cc:1398] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:47.995663 10447 tablet_bootstrap.cc:492] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e: Bootstrap starting.
I20240502 20:46:47.996380 10447 tablet_bootstrap.cc:654] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:47.997244 10447 tablet_bootstrap.cc:492] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e: No bootstrap required, opened a new log
I20240502 20:46:47.997334 10447 ts_tablet_manager.cc:1367] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:47.997716 10447 raft_consensus.cc:348] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:47.997823 10447 raft_consensus.cc:374] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:47.997864 10447 raft_consensus.cc:729] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Initialized, Role: FOLLOWER
I20240502 20:46:47.997988 10447 consensus_queue.cc:260] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:47.998239 10447 ts_tablet_manager.cc:1398] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:48.106705 10293 raft_consensus.cc:482] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:48.106837 10293 raft_consensus.cc:504] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:48.107411 10293 leader_election.cc:288] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:48.107702 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" is_pre_election: true
I20240502 20:46:48.107834 9289 raft_consensus.cc:2454] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 0.
I20240502 20:46:48.108050 9556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" is_pre_election: true
I20240502 20:46:48.108194 9556 raft_consensus.cc:2454] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 0.
I20240502 20:46:48.108492 9357 leader_election.cc:302] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:48.108660 10293 raft_consensus.cc:2781] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:48.108703 10293 raft_consensus.cc:482] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:48.108739 10293 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:48.109477 10293 raft_consensus.cc:504] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:48.109795 10293 leader_election.cc:288] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:48.110090 9556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:48.110185 9556 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:48.110265 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:48.110373 9289 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:48.111063 9556 raft_consensus.cc:2454] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 1.
I20240502 20:46:48.111302 9357 leader_election.cc:302] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:48.111356 9289 raft_consensus.cc:2454] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 1.
I20240502 20:46:48.111482 10293 raft_consensus.cc:2781] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:48.111575 10293 raft_consensus.cc:686] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 1 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:48.111707 10293 consensus_queue.cc:237] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:48.113307 9140 catalog_manager.cc:5484] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 1 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:48.129536 10438 raft_consensus.cc:482] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:48.129643 10438 raft_consensus.cc:504] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:48.130129 10438 leader_election.cc:288] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:48.130600 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64136bd82a30486e9d56fc28da4bd448" is_pre_election: true
I20240502 20:46:48.130753 9688 raft_consensus.cc:2454] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 0.
I20240502 20:46:48.131121 9492 leader_election.cc:302] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:48.131351 10438 raft_consensus.cc:2781] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:48.131400 10438 raft_consensus.cc:482] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:48.131438 10438 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:48.132369 10438 raft_consensus.cc:504] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:48.132721 10438 leader_election.cc:288] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 1 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:48.133234 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:48.133361 9688 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:48.133893 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:48.134014 9424 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:48.134142 9688 raft_consensus.cc:2454] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 1.
I20240502 20:46:48.134613 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc" is_pre_election: true
I20240502 20:46:48.135087 9424 raft_consensus.cc:2454] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 1.
I20240502 20:46:48.135495 9492 leader_election.cc:302] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:48.135743 10438 raft_consensus.cc:2781] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:48.135893 10438 raft_consensus.cc:686] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:48.136029 10438 consensus_queue.cc:237] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:48.137530 9143 catalog_manager.cc:5484] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 1 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:48.216912 10438 raft_consensus.cc:482] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:48.217015 10438 raft_consensus.cc:504] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:48.217342 10438 leader_election.cc:288] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:48.217712 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64136bd82a30486e9d56fc28da4bd448" is_pre_election: true
I20240502 20:46:48.217761 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" is_pre_election: true
I20240502 20:46:48.217851 9688 raft_consensus.cc:2454] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 0.
I20240502 20:46:48.217877 9289 raft_consensus.cc:2454] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 0.
I20240502 20:46:48.218130 9490 leader_election.cc:302] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:48.218278 10438 raft_consensus.cc:2781] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:48.218338 10438 raft_consensus.cc:482] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:48.218374 10438 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:48.219197 10438 raft_consensus.cc:504] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:48.219497 10438 leader_election.cc:288] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 1 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:48.219785 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:48.219799 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:48.219879 9688 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:48.219906 9289 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:48.220531 9688 raft_consensus.cc:2454] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 1.
I20240502 20:46:48.220652 9289 raft_consensus.cc:2454] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 1.
I20240502 20:46:48.220777 9492 leader_election.cc:302] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:48.220921 10438 raft_consensus.cc:2781] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:48.220990 10438 raft_consensus.cc:686] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:48.221114 10438 consensus_queue.cc:237] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:48.222563 9143 catalog_manager.cc:5484] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 1 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
W20240502 20:46:48.374639 10520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:48.374876 10520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:48.378003 10520 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
I20240502 20:46:48.464033 9289 raft_consensus.cc:1261] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:48.464589 10438 consensus_queue.cc:1035] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:48.465549 9688 raft_consensus.cc:1261] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:48.466080 10438 consensus_queue.cc:1035] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:48.469008 9424 raft_consensus.cc:1261] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:48.469475 9556 raft_consensus.cc:1261] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:48.470016 9689 raft_consensus.cc:1261] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:48.470631 10293 consensus_queue.cc:1035] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:48.471504 9288 raft_consensus.cc:1261] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:48.473429 10517 consensus_queue.cc:1035] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:48.473783 10517 consensus_queue.cc:1035] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:48.477319 10462 consensus_queue.cc:1035] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 108.403 ms
time per row: 0.0542015 ms
I20240502 20:46:48.529453 9138 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33288:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:48.535056 9268 tablet_service.cc:1467] Processing CreateTablet for tablet da71e448ed304b338946568c4ee94fe6 (DEFAULT_TABLE table=rebalance_test_table_4 [id=5d1fcff1058e4c519df2822407c28eb9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:48.535398 9268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da71e448ed304b338946568c4ee94fe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:48.535941 9404 tablet_service.cc:1467] Processing CreateTablet for tablet da71e448ed304b338946568c4ee94fe6 (DEFAULT_TABLE table=rebalance_test_table_4 [id=5d1fcff1058e4c519df2822407c28eb9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:48.536209 9404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da71e448ed304b338946568c4ee94fe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:48.535980 9402 tablet_service.cc:1467] Processing CreateTablet for tablet 9db841835d2e4b50b149c0980e762fac (DEFAULT_TABLE table=rebalance_test_table_4 [id=5d1fcff1058e4c519df2822407c28eb9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:48.536449 9402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9db841835d2e4b50b149c0980e762fac. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:48.536422 9403 tablet_service.cc:1467] Processing CreateTablet for tablet d25d4ef404e34234a1a50da88deb5f90 (DEFAULT_TABLE table=rebalance_test_table_4 [id=5d1fcff1058e4c519df2822407c28eb9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:48.536655 9403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d25d4ef404e34234a1a50da88deb5f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:48.538389 9668 tablet_service.cc:1467] Processing CreateTablet for tablet 9db841835d2e4b50b149c0980e762fac (DEFAULT_TABLE table=rebalance_test_table_4 [id=5d1fcff1058e4c519df2822407c28eb9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:48.538619 9668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9db841835d2e4b50b149c0980e762fac. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:48.538915 9535 tablet_service.cc:1467] Processing CreateTablet for tablet da71e448ed304b338946568c4ee94fe6 (DEFAULT_TABLE table=rebalance_test_table_4 [id=5d1fcff1058e4c519df2822407c28eb9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:48.538957 9536 tablet_service.cc:1467] Processing CreateTablet for tablet d25d4ef404e34234a1a50da88deb5f90 (DEFAULT_TABLE table=rebalance_test_table_4 [id=5d1fcff1058e4c519df2822407c28eb9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:48.539144 9535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da71e448ed304b338946568c4ee94fe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:48.539753 9536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d25d4ef404e34234a1a50da88deb5f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:48.540762 9667 tablet_service.cc:1467] Processing CreateTablet for tablet d25d4ef404e34234a1a50da88deb5f90 (DEFAULT_TABLE table=rebalance_test_table_4 [id=5d1fcff1058e4c519df2822407c28eb9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:48.540977 9667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d25d4ef404e34234a1a50da88deb5f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:48.542030 9268 tablet_service.cc:1467] Processing CreateTablet for tablet 9db841835d2e4b50b149c0980e762fac (DEFAULT_TABLE table=rebalance_test_table_4 [id=5d1fcff1058e4c519df2822407c28eb9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:48.542249 9268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9db841835d2e4b50b149c0980e762fac. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:48.542655 10582 tablet_bootstrap.cc:492] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc: Bootstrap starting.
I20240502 20:46:48.543527 10582 tablet_bootstrap.cc:654] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:48.543838 10583 tablet_bootstrap.cc:492] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448: Bootstrap starting.
I20240502 20:46:48.544589 10583 tablet_bootstrap.cc:654] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:48.545153 10584 tablet_bootstrap.cc:492] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e: Bootstrap starting.
I20240502 20:46:48.545249 10581 tablet_bootstrap.cc:492] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f: Bootstrap starting.
I20240502 20:46:48.545827 10584 tablet_bootstrap.cc:654] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:48.546124 10581 tablet_bootstrap.cc:654] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:48.546641 10583 tablet_bootstrap.cc:492] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448: No bootstrap required, opened a new log
I20240502 20:46:48.546731 10583 ts_tablet_manager.cc:1367] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:48.547176 10581 tablet_bootstrap.cc:492] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f: No bootstrap required, opened a new log
I20240502 20:46:48.547171 10583 raft_consensus.cc:348] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:48.547257 10581 ts_tablet_manager.cc:1367] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:48.547266 10583 raft_consensus.cc:374] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:48.547297 10583 raft_consensus.cc:729] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Initialized, Role: FOLLOWER
I20240502 20:46:48.547477 10584 tablet_bootstrap.cc:492] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e: No bootstrap required, opened a new log
I20240502 20:46:48.547430 10583 consensus_queue.cc:260] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:48.547545 10584 ts_tablet_manager.cc:1367] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:48.547683 10581 raft_consensus.cc:348] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:48.547755 10583 ts_tablet_manager.cc:1398] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:48.547777 10581 raft_consensus.cc:374] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:48.547807 10581 raft_consensus.cc:729] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Initialized, Role: FOLLOWER
I20240502 20:46:48.547904 10583 tablet_bootstrap.cc:492] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448: Bootstrap starting.
I20240502 20:46:48.547909 10584 raft_consensus.cc:348] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:48.547920 10581 consensus_queue.cc:260] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:48.547992 10584 raft_consensus.cc:374] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:48.548022 10584 raft_consensus.cc:729] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Initialized, Role: FOLLOWER
I20240502 20:46:48.548126 10584 consensus_queue.cc:260] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:48.548177 10581 ts_tablet_manager.cc:1398] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:48.548336 10581 tablet_bootstrap.cc:492] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f: Bootstrap starting.
I20240502 20:46:48.548384 10584 ts_tablet_manager.cc:1398] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:48.548540 10584 tablet_bootstrap.cc:492] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e: Bootstrap starting.
I20240502 20:46:48.548727 10583 tablet_bootstrap.cc:654] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:48.549160 10581 tablet_bootstrap.cc:654] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:48.550163 10584 tablet_bootstrap.cc:654] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:48.551120 10581 tablet_bootstrap.cc:492] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f: No bootstrap required, opened a new log
I20240502 20:46:48.551221 10581 ts_tablet_manager.cc:1367] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:48.551666 10581 raft_consensus.cc:348] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:48.551774 10581 raft_consensus.cc:374] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:48.551810 10581 raft_consensus.cc:729] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Initialized, Role: FOLLOWER
I20240502 20:46:48.551936 10581 consensus_queue.cc:260] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:48.552251 10581 ts_tablet_manager.cc:1398] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:48.552547 10582 tablet_bootstrap.cc:492] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc: No bootstrap required, opened a new log
I20240502 20:46:48.552641 10582 ts_tablet_manager.cc:1367] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 20:46:48.553064 10583 tablet_bootstrap.cc:492] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448: No bootstrap required, opened a new log
I20240502 20:46:48.553054 10582 raft_consensus.cc:348] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:48.553136 10582 raft_consensus.cc:374] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:48.553169 10582 raft_consensus.cc:729] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Initialized, Role: FOLLOWER
I20240502 20:46:48.553175 10583 ts_tablet_manager.cc:1367] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 20:46:48.553282 10582 consensus_queue.cc:260] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:48.553577 10583 raft_consensus.cc:348] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:48.553668 10583 raft_consensus.cc:374] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:48.553699 10583 raft_consensus.cc:729] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Initialized, Role: FOLLOWER
I20240502 20:46:48.553802 10583 consensus_queue.cc:260] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:48.553884 10582 ts_tablet_manager.cc:1398] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:48.554062 10582 tablet_bootstrap.cc:492] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc: Bootstrap starting.
I20240502 20:46:48.554567 10583 ts_tablet_manager.cc:1398] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:48.554993 10582 tablet_bootstrap.cc:654] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:48.556252 10584 tablet_bootstrap.cc:492] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e: No bootstrap required, opened a new log
I20240502 20:46:48.556350 10584 ts_tablet_manager.cc:1367] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20240502 20:46:48.556820 10584 raft_consensus.cc:348] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:48.556933 10584 raft_consensus.cc:374] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:48.556977 10584 raft_consensus.cc:729] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Initialized, Role: FOLLOWER
I20240502 20:46:48.557106 10584 consensus_queue.cc:260] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:48.557356 10584 ts_tablet_manager.cc:1398] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:48.559056 10582 tablet_bootstrap.cc:492] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc: No bootstrap required, opened a new log
I20240502 20:46:48.559149 10582 ts_tablet_manager.cc:1367] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240502 20:46:48.559505 10582 raft_consensus.cc:348] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:48.559598 10582 raft_consensus.cc:374] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:48.559628 10582 raft_consensus.cc:729] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Initialized, Role: FOLLOWER
I20240502 20:46:48.559732 10582 consensus_queue.cc:260] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:48.559949 10582 ts_tablet_manager.cc:1398] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:48.560092 10582 tablet_bootstrap.cc:492] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc: Bootstrap starting.
I20240502 20:46:48.561017 10582 tablet_bootstrap.cc:654] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:48.561980 10582 tablet_bootstrap.cc:492] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc: No bootstrap required, opened a new log
I20240502 20:46:48.562055 10582 ts_tablet_manager.cc:1367] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:48.562381 10582 raft_consensus.cc:348] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:48.562501 10582 raft_consensus.cc:374] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:48.562537 10582 raft_consensus.cc:729] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Initialized, Role: FOLLOWER
I20240502 20:46:48.562634 10582 consensus_queue.cc:260] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:48.562840 10582 ts_tablet_manager.cc:1398] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:48.575876 10462 raft_consensus.cc:482] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:48.575969 10462 raft_consensus.cc:504] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:48.576274 10462 leader_election.cc:288] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:48.576575 9288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" is_pre_election: true
I20240502 20:46:48.576587 9556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" is_pre_election: true
I20240502 20:46:48.576676 9288 raft_consensus.cc:2454] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 0.
I20240502 20:46:48.576699 9556 raft_consensus.cc:2454] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 0.
I20240502 20:46:48.576934 9355 leader_election.cc:302] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:48.577104 10462 raft_consensus.cc:2781] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:48.577164 10462 raft_consensus.cc:482] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:48.577214 10462 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:48.577975 10462 raft_consensus.cc:504] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:48.578291 10462 leader_election.cc:288] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:48.578613 9288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:48.578626 9556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:48.578723 9556 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:48.578723 9288 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:48.579545 9288 raft_consensus.cc:2454] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 1.
I20240502 20:46:48.579545 9556 raft_consensus.cc:2454] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 1.
I20240502 20:46:48.579818 9357 leader_election.cc:302] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:48.579982 10462 raft_consensus.cc:2781] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:48.580070 10462 raft_consensus.cc:686] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:48.580196 10462 consensus_queue.cc:237] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:48.581666 9138 catalog_manager.cc:5484] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 1 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:48.644810 10302 raft_consensus.cc:482] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:48.644904 10302 raft_consensus.cc:504] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:48.645195 10302 leader_election.cc:288] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:48.645519 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc" is_pre_election: true
I20240502 20:46:48.645540 9288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" is_pre_election: true
I20240502 20:46:48.645637 9424 raft_consensus.cc:2454] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 0.
I20240502 20:46:48.645653 9288 raft_consensus.cc:2454] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 0.
I20240502 20:46:48.645893 9622 leader_election.cc:302] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:48.646070 10302 raft_consensus.cc:2781] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:48.646123 10302 raft_consensus.cc:482] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:48.646159 10302 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:48.646886 10302 raft_consensus.cc:504] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:48.647176 10302 leader_election.cc:288] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:48.647497 9288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:48.647497 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:48.647624 9288 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:48.647622 9424 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:48.648525 9288 raft_consensus.cc:2454] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 1.
I20240502 20:46:48.648525 9424 raft_consensus.cc:2454] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 1.
I20240502 20:46:48.648809 9622 leader_election.cc:302] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:48.649024 10302 raft_consensus.cc:2781] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:48.649245 10302 raft_consensus.cc:686] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 1 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:48.649394 10302 consensus_queue.cc:237] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:48.651043 9138 catalog_manager.cc:5484] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:48.666513 10554 raft_consensus.cc:482] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:48.666620 10554 raft_consensus.cc:504] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:48.666999 10554 leader_election.cc:288] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:48.667279 9689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64136bd82a30486e9d56fc28da4bd448" is_pre_election: true
I20240502 20:46:48.667308 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc" is_pre_election: true
I20240502 20:46:48.667393 9689 raft_consensus.cc:2454] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 0.
I20240502 20:46:48.667414 9424 raft_consensus.cc:2454] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 0.
I20240502 20:46:48.667654 9492 leader_election.cc:302] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:48.667815 10554 raft_consensus.cc:2781] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:48.667876 10554 raft_consensus.cc:482] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:48.667912 10554 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:48.668736 10554 raft_consensus.cc:504] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:48.669039 10554 leader_election.cc:288] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 1 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:48.669360 9689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:48.669390 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:48.669459 9689 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:48.669488 9424 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:48.670185 9424 raft_consensus.cc:2454] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 1.
I20240502 20:46:48.670218 9689 raft_consensus.cc:2454] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 1.
I20240502 20:46:48.670508 9492 leader_election.cc:302] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:48.670665 10554 raft_consensus.cc:2781] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:48.670763 10554 raft_consensus.cc:686] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:48.670912 10554 consensus_queue.cc:237] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:48.672330 9138 catalog_manager.cc:5484] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 1 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
W20240502 20:46:48.797796 10599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:48.797994 10599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:48.800911 10599 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
I20240502 20:46:48.874730 9288 raft_consensus.cc:1261] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:48.875159 9689 raft_consensus.cc:1261] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:48.875620 9424 raft_consensus.cc:1261] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:48.875805 9556 raft_consensus.cc:1261] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:48.875929 10554 consensus_queue.cc:1035] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:48.876130 9287 raft_consensus.cc:1261] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:48.876422 10554 consensus_queue.cc:1035] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:48.876528 10462 consensus_queue.cc:1035] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:48.876682 10293 consensus_queue.cc:1035] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:48.877027 10302 consensus_queue.cc:1035] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:48.893066 9423 raft_consensus.cc:1261] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:48.893721 10596 consensus_queue.cc:1035] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 102.545 ms
time per row: 0.0512727 ms
I20240502 20:46:48.941107 9139 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33288:
name: "rebalance_test_table_5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:48.945916 9268 tablet_service.cc:1467] Processing CreateTablet for tablet 867c2c1a772f4133a2a83e1e9ea2a831 (DEFAULT_TABLE table=rebalance_test_table_5 [id=d3429591479c432588c0127caf0a3689]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:48.945950 9536 tablet_service.cc:1467] Processing CreateTablet for tablet 867c2c1a772f4133a2a83e1e9ea2a831 (DEFAULT_TABLE table=rebalance_test_table_5 [id=d3429591479c432588c0127caf0a3689]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:48.946213 9268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 867c2c1a772f4133a2a83e1e9ea2a831. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:48.946223 9536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 867c2c1a772f4133a2a83e1e9ea2a831. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:48.947784 10581 tablet_bootstrap.cc:492] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f: Bootstrap starting.
I20240502 20:46:48.947821 10584 tablet_bootstrap.cc:492] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e: Bootstrap starting.
I20240502 20:46:48.948051 9667 tablet_service.cc:1467] Processing CreateTablet for tablet 867c2c1a772f4133a2a83e1e9ea2a831 (DEFAULT_TABLE table=rebalance_test_table_5 [id=d3429591479c432588c0127caf0a3689]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:48.948179 9668 tablet_service.cc:1467] Processing CreateTablet for tablet b59d99b81d3d4528a74c045f6ca41199 (DEFAULT_TABLE table=rebalance_test_table_5 [id=d3429591479c432588c0127caf0a3689]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:48.948290 9667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 867c2c1a772f4133a2a83e1e9ea2a831. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:48.948400 9668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b59d99b81d3d4528a74c045f6ca41199. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:48.948621 10584 tablet_bootstrap.cc:654] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:48.948669 10581 tablet_bootstrap.cc:654] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:48.949731 10581 tablet_bootstrap.cc:492] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f: No bootstrap required, opened a new log
I20240502 20:46:48.949797 10581 ts_tablet_manager.cc:1367] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:48.950073 10583 tablet_bootstrap.cc:492] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448: Bootstrap starting.
I20240502 20:46:48.950196 10581 raft_consensus.cc:348] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:48.950299 10581 raft_consensus.cc:374] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:48.950330 10581 raft_consensus.cc:729] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Initialized, Role: FOLLOWER
I20240502 20:46:48.950431 10581 consensus_queue.cc:260] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:48.950738 10581 ts_tablet_manager.cc:1398] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:48.950881 9536 tablet_service.cc:1467] Processing CreateTablet for tablet b59d99b81d3d4528a74c045f6ca41199 (DEFAULT_TABLE table=rebalance_test_table_5 [id=d3429591479c432588c0127caf0a3689]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:48.950946 10584 tablet_bootstrap.cc:492] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e: No bootstrap required, opened a new log
I20240502 20:46:48.951006 10584 ts_tablet_manager.cc:1367] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 20:46:48.951083 9536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b59d99b81d3d4528a74c045f6ca41199. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:48.951186 9269 tablet_service.cc:1467] Processing CreateTablet for tablet 20698b2531674eefb9df3125a86653ba (DEFAULT_TABLE table=rebalance_test_table_5 [id=d3429591479c432588c0127caf0a3689]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:48.951393 9269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20698b2531674eefb9df3125a86653ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:48.951413 10584 raft_consensus.cc:348] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:48.951498 10584 raft_consensus.cc:374] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:48.951522 10584 raft_consensus.cc:729] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Initialized, Role: FOLLOWER
I20240502 20:46:48.951527 9268 tablet_service.cc:1467] Processing CreateTablet for tablet b59d99b81d3d4528a74c045f6ca41199 (DEFAULT_TABLE table=rebalance_test_table_5 [id=d3429591479c432588c0127caf0a3689]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:48.951622 10584 consensus_queue.cc:260] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:48.951709 9268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b59d99b81d3d4528a74c045f6ca41199. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:48.951860 10584 ts_tablet_manager.cc:1398] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:48.952416 9535 tablet_service.cc:1467] Processing CreateTablet for tablet 20698b2531674eefb9df3125a86653ba (DEFAULT_TABLE table=rebalance_test_table_5 [id=d3429591479c432588c0127caf0a3689]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:48.952625 9535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20698b2531674eefb9df3125a86653ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:48.953208 10584 tablet_bootstrap.cc:492] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e: Bootstrap starting.
I20240502 20:46:48.953408 9403 tablet_service.cc:1467] Processing CreateTablet for tablet 20698b2531674eefb9df3125a86653ba (DEFAULT_TABLE table=rebalance_test_table_5 [id=d3429591479c432588c0127caf0a3689]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:48.953680 9403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20698b2531674eefb9df3125a86653ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:48.954036 10584 tablet_bootstrap.cc:654] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:48.954119 10581 tablet_bootstrap.cc:492] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f: Bootstrap starting.
I20240502 20:46:48.954919 10581 tablet_bootstrap.cc:654] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:48.955730 10582 tablet_bootstrap.cc:492] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc: Bootstrap starting.
I20240502 20:46:48.955889 10583 tablet_bootstrap.cc:654] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:48.956503 10582 tablet_bootstrap.cc:654] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:48.956949 10581 tablet_bootstrap.cc:492] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f: No bootstrap required, opened a new log
I20240502 20:46:48.957039 10581 ts_tablet_manager.cc:1367] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 20:46:48.957398 10583 tablet_bootstrap.cc:492] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448: No bootstrap required, opened a new log
I20240502 20:46:48.957413 10581 raft_consensus.cc:348] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:48.957489 10583 ts_tablet_manager.cc:1367] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 20:46:48.957515 10581 raft_consensus.cc:374] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:48.957561 10582 tablet_bootstrap.cc:492] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc: No bootstrap required, opened a new log
I20240502 20:46:48.957562 10581 raft_consensus.cc:729] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Initialized, Role: FOLLOWER
I20240502 20:46:48.957638 10582 ts_tablet_manager.cc:1367] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:48.957680 10581 consensus_queue.cc:260] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:48.957906 10581 ts_tablet_manager.cc:1398] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:48.957890 10583 raft_consensus.cc:348] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:48.958001 10583 raft_consensus.cc:374] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:48.958036 10583 raft_consensus.cc:729] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Initialized, Role: FOLLOWER
I20240502 20:46:48.958004 10582 raft_consensus.cc:348] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:48.958057 10581 tablet_bootstrap.cc:492] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f: Bootstrap starting.
I20240502 20:46:48.958088 10582 raft_consensus.cc:374] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:48.958115 10582 raft_consensus.cc:729] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Initialized, Role: FOLLOWER
I20240502 20:46:48.958143 10583 consensus_queue.cc:260] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:48.958213 10582 consensus_queue.cc:260] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:48.958380 10583 ts_tablet_manager.cc:1398] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:48.958453 10582 ts_tablet_manager.cc:1398] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:48.958559 10583 tablet_bootstrap.cc:492] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448: Bootstrap starting.
I20240502 20:46:48.959192 10581 tablet_bootstrap.cc:654] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:48.959630 10583 tablet_bootstrap.cc:654] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:48.959964 10584 tablet_bootstrap.cc:492] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e: No bootstrap required, opened a new log
I20240502 20:46:48.960055 10584 ts_tablet_manager.cc:1367] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20240502 20:46:48.960418 10584 raft_consensus.cc:348] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:48.960510 10584 raft_consensus.cc:374] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:48.960539 10584 raft_consensus.cc:729] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Initialized, Role: FOLLOWER
I20240502 20:46:48.960638 10584 consensus_queue.cc:260] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:48.960393 10581 tablet_bootstrap.cc:492] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f: No bootstrap required, opened a new log
I20240502 20:46:48.960883 10581 ts_tablet_manager.cc:1367] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:48.960908 10584 ts_tablet_manager.cc:1398] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:48.961064 10584 tablet_bootstrap.cc:492] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e: Bootstrap starting.
I20240502 20:46:48.961241 10581 raft_consensus.cc:348] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:48.961839 10584 tablet_bootstrap.cc:654] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:48.961336 10581 raft_consensus.cc:374] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:48.962431 10581 raft_consensus.cc:729] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Initialized, Role: FOLLOWER
I20240502 20:46:48.962517 10583 tablet_bootstrap.cc:492] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448: No bootstrap required, opened a new log
I20240502 20:46:48.962589 10583 ts_tablet_manager.cc:1367] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:48.962603 10581 consensus_queue.cc:260] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:48.962952 10581 ts_tablet_manager.cc:1398] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:48.962975 10583 raft_consensus.cc:348] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:48.963065 10583 raft_consensus.cc:374] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:48.963094 10583 raft_consensus.cc:729] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Initialized, Role: FOLLOWER
I20240502 20:46:48.963193 10583 consensus_queue.cc:260] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:48.963431 10583 ts_tablet_manager.cc:1398] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:48.963652 10584 tablet_bootstrap.cc:492] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e: No bootstrap required, opened a new log
I20240502 20:46:48.963728 10584 ts_tablet_manager.cc:1367] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:48.964120 10584 raft_consensus.cc:348] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:48.964257 10584 raft_consensus.cc:374] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:48.964290 10584 raft_consensus.cc:729] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Initialized, Role: FOLLOWER
I20240502 20:46:48.964391 10584 consensus_queue.cc:260] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:48.964634 10584 ts_tablet_manager.cc:1398] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:49.197687 10442 raft_consensus.cc:482] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:49.197816 10442 raft_consensus.cc:504] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:49.198194 10442 leader_election.cc:288] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:49.198542 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20698b2531674eefb9df3125a86653ba" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc" is_pre_election: true
I20240502 20:46:49.198567 9556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20698b2531674eefb9df3125a86653ba" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" is_pre_election: true
I20240502 20:46:49.198693 9556 raft_consensus.cc:2454] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 0.
I20240502 20:46:49.198693 9423 raft_consensus.cc:2454] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 0.
I20240502 20:46:49.198993 9223 leader_election.cc:302] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:49.199174 10442 raft_consensus.cc:2781] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:49.199251 10442 raft_consensus.cc:482] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:49.199280 10442 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:49.200065 10442 raft_consensus.cc:504] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:49.200377 10442 leader_election.cc:288] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 1 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:49.200654 9556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20698b2531674eefb9df3125a86653ba" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:49.200649 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20698b2531674eefb9df3125a86653ba" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:49.200763 9556 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:49.200763 9423 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:49.201455 9556 raft_consensus.cc:2454] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 1.
I20240502 20:46:49.201539 9423 raft_consensus.cc:2454] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 1.
I20240502 20:46:49.201716 9223 leader_election.cc:302] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:49.201896 10442 raft_consensus.cc:2781] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:49.202056 10442 raft_consensus.cc:686] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:49.202186 10442 consensus_queue.cc:237] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:49.203716 9141 catalog_manager.cc:5484] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 1 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:49.227826 10442 raft_consensus.cc:482] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:49.227907 10442 raft_consensus.cc:504] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:49.228245 10442 leader_election.cc:288] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:49.228524 9556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" is_pre_election: true
I20240502 20:46:49.228653 9556 raft_consensus.cc:2454] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 0.
I20240502 20:46:49.228884 9223 leader_election.cc:302] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:49.229053 10442 raft_consensus.cc:2781] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:49.229118 10442 raft_consensus.cc:482] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:49.229157 10442 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:49.231503 9689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64136bd82a30486e9d56fc28da4bd448" is_pre_election: true
I20240502 20:46:49.231617 9689 raft_consensus.cc:2454] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 0.
I20240502 20:46:49.247125 10442 raft_consensus.cc:504] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:49.247419 10442 leader_election.cc:288] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 1 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:49.247746 9556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:49.247771 9689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:49.247876 9689 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:49.247876 9556 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:49.248739 9556 raft_consensus.cc:2454] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 1.
I20240502 20:46:49.248739 9689 raft_consensus.cc:2454] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 1.
I20240502 20:46:49.249008 9223 leader_election.cc:302] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:49.249159 10442 raft_consensus.cc:2781] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:49.249266 10442 raft_consensus.cc:686] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:49.249398 10442 consensus_queue.cc:237] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:49.250869 9141 catalog_manager.cc:5484] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 1 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:49.290412 10636 raft_consensus.cc:482] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:49.290540 10636 raft_consensus.cc:504] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:49.290913 10636 leader_election.cc:288] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:49.291244 9288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" is_pre_election: true
I20240502 20:46:49.291251 9689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64136bd82a30486e9d56fc28da4bd448" is_pre_election: true
I20240502 20:46:49.291373 9288 raft_consensus.cc:2454] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 0.
I20240502 20:46:49.291373 9689 raft_consensus.cc:2454] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 0.
I20240502 20:46:49.291628 9492 leader_election.cc:302] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:49.291806 10636 raft_consensus.cc:2781] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:49.291863 10636 raft_consensus.cc:482] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:49.291903 10636 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:49.292605 10636 raft_consensus.cc:504] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:49.292908 10636 leader_election.cc:288] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 1 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:49.293201 9288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:49.293207 9689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:49.293289 9689 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:49.293296 9288 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:49.293963 9689 raft_consensus.cc:2454] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 1.
I20240502 20:46:49.293967 9288 raft_consensus.cc:2454] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 1.
I20240502 20:46:49.294219 9492 leader_election.cc:302] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:49.294392 10636 raft_consensus.cc:2781] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:49.294519 10636 raft_consensus.cc:686] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:49.294652 10636 consensus_queue.cc:237] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:49.296175 9141 catalog_manager.cc:5484] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 1 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
W20240502 20:46:49.420706 10651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:49.420898 10651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:49.424165 10651 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_5'
I20240502 20:46:49.497334 9689 raft_consensus.cc:1261] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:49.497689 9288 raft_consensus.cc:1261] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:49.498319 10554 consensus_queue.cc:1035] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:49.498543 10636 consensus_queue.cc:1035] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:49.501875 9556 raft_consensus.cc:1261] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:49.502562 10646 consensus_queue.cc:1035] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:49.502795 9423 raft_consensus.cc:1261] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:49.503182 10646 consensus_queue.cc:1035] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:49.503227 9688 raft_consensus.cc:1261] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:49.504926 10442 consensus_queue.cc:1035] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:49.505389 9556 raft_consensus.cc:1261] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:49.507778 10442 consensus_queue.cc:1035] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:49.510838 10685 mvcc.cc:205] Tried to move back new op lower bound from 7023340787696115712 to 7023340786872094720. Current Snapshot: MvccSnapshot[applied={T|T < 7023340787696115712}]
I20240502 20:46:49.520685 10682 mvcc.cc:205] Tried to move back new op lower bound from 7023340787710795776 to 7023340786872094720. Current Snapshot: MvccSnapshot[applied={T|T < 7023340787696115712}]
I20240502 20:46:49.527635 10686 mvcc.cc:205] Tried to move back new op lower bound from 7023340787718762496 to 7023340786686787584. Current Snapshot: MvccSnapshot[applied={T|T < 7023340787718762496}]
INSERT report
rows total: 2000
time total: 99.2091 ms
time per row: 0.0496045 ms
I20240502 20:46:49.564955 9142 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33288:
name: "rebalance_test_table_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:49.570025 9403 tablet_service.cc:1467] Processing CreateTablet for tablet cd36ed4462ca494c87169bd2ca7add45 (DEFAULT_TABLE table=rebalance_test_table_6 [id=632304afe23f467e8cc404c7b307faeb]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:49.570096 9668 tablet_service.cc:1467] Processing CreateTablet for tablet cd36ed4462ca494c87169bd2ca7add45 (DEFAULT_TABLE table=rebalance_test_table_6 [id=632304afe23f467e8cc404c7b307faeb]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:49.570309 9403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd36ed4462ca494c87169bd2ca7add45. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:49.570318 9668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd36ed4462ca494c87169bd2ca7add45. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:49.570696 9402 tablet_service.cc:1467] Processing CreateTablet for tablet dea141ae57324a879dfd26cbeea9aead (DEFAULT_TABLE table=rebalance_test_table_6 [id=632304afe23f467e8cc404c7b307faeb]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:49.570930 9402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dea141ae57324a879dfd26cbeea9aead. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:49.570696 9404 tablet_service.cc:1467] Processing CreateTablet for tablet c17aac2ef2a74cf6a6f92f84b908f40e (DEFAULT_TABLE table=rebalance_test_table_6 [id=632304afe23f467e8cc404c7b307faeb]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:49.571074 9404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c17aac2ef2a74cf6a6f92f84b908f40e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:49.571168 9268 tablet_service.cc:1467] Processing CreateTablet for tablet cd36ed4462ca494c87169bd2ca7add45 (DEFAULT_TABLE table=rebalance_test_table_6 [id=632304afe23f467e8cc404c7b307faeb]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:49.571254 9269 tablet_service.cc:1467] Processing CreateTablet for tablet dea141ae57324a879dfd26cbeea9aead (DEFAULT_TABLE table=rebalance_test_table_6 [id=632304afe23f467e8cc404c7b307faeb]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:49.571424 9268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd36ed4462ca494c87169bd2ca7add45. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:49.571554 9269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dea141ae57324a879dfd26cbeea9aead. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:49.571645 9667 tablet_service.cc:1467] Processing CreateTablet for tablet c17aac2ef2a74cf6a6f92f84b908f40e (DEFAULT_TABLE table=rebalance_test_table_6 [id=632304afe23f467e8cc404c7b307faeb]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:49.571877 9667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c17aac2ef2a74cf6a6f92f84b908f40e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:49.572301 9669 tablet_service.cc:1467] Processing CreateTablet for tablet dea141ae57324a879dfd26cbeea9aead (DEFAULT_TABLE table=rebalance_test_table_6 [id=632304afe23f467e8cc404c7b307faeb]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:49.572530 9669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dea141ae57324a879dfd26cbeea9aead. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:49.573220 9535 tablet_service.cc:1467] Processing CreateTablet for tablet c17aac2ef2a74cf6a6f92f84b908f40e (DEFAULT_TABLE table=rebalance_test_table_6 [id=632304afe23f467e8cc404c7b307faeb]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:49.573469 9535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c17aac2ef2a74cf6a6f92f84b908f40e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:49.574745 10709 tablet_bootstrap.cc:492] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f: Bootstrap starting.
I20240502 20:46:49.575076 10710 tablet_bootstrap.cc:492] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e: Bootstrap starting.
I20240502 20:46:49.575594 10709 tablet_bootstrap.cc:654] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:49.575850 10710 tablet_bootstrap.cc:654] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:49.576640 10709 tablet_bootstrap.cc:492] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f: No bootstrap required, opened a new log
I20240502 20:46:49.576737 10709 ts_tablet_manager.cc:1367] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:49.577131 10709 raft_consensus.cc:348] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:49.577217 10709 raft_consensus.cc:374] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:49.577245 10709 raft_consensus.cc:729] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Initialized, Role: FOLLOWER
I20240502 20:46:49.577375 10709 consensus_queue.cc:260] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:49.577616 10709 ts_tablet_manager.cc:1398] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:49.577777 10709 tablet_bootstrap.cc:492] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f: Bootstrap starting.
I20240502 20:46:49.578692 10709 tablet_bootstrap.cc:654] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:49.579228 10710 tablet_bootstrap.cc:492] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e: No bootstrap required, opened a new log
I20240502 20:46:49.579322 10710 ts_tablet_manager.cc:1367] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:49.579763 10710 raft_consensus.cc:348] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:49.579881 10710 raft_consensus.cc:374] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:49.579919 10710 raft_consensus.cc:729] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Initialized, Role: FOLLOWER
I20240502 20:46:49.580041 10710 consensus_queue.cc:260] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:49.580216 10709 tablet_bootstrap.cc:492] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f: No bootstrap required, opened a new log
I20240502 20:46:49.580286 10710 ts_tablet_manager.cc:1398] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:49.580314 10709 ts_tablet_manager.cc:1367] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:49.580699 10709 raft_consensus.cc:348] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:49.580783 10709 raft_consensus.cc:374] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:49.580811 10709 raft_consensus.cc:729] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Initialized, Role: FOLLOWER
I20240502 20:46:49.580927 10709 consensus_queue.cc:260] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:49.581183 10709 ts_tablet_manager.cc:1398] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:49.596549 10716 tablet_bootstrap.cc:492] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc: Bootstrap starting.
I20240502 20:46:49.597352 10716 tablet_bootstrap.cc:654] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:49.598073 10717 tablet_bootstrap.cc:492] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448: Bootstrap starting.
I20240502 20:46:49.598351 10716 tablet_bootstrap.cc:492] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc: No bootstrap required, opened a new log
I20240502 20:46:49.598434 10716 ts_tablet_manager.cc:1367] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:49.598806 10716 raft_consensus.cc:348] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:49.598896 10716 raft_consensus.cc:374] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:49.598908 10717 tablet_bootstrap.cc:654] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:49.598924 10716 raft_consensus.cc:729] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Initialized, Role: FOLLOWER
I20240502 20:46:49.599037 10716 consensus_queue.cc:260] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:49.599270 10716 ts_tablet_manager.cc:1398] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:49.599450 10716 tablet_bootstrap.cc:492] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc: Bootstrap starting.
I20240502 20:46:49.599862 10717 tablet_bootstrap.cc:492] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448: No bootstrap required, opened a new log
I20240502 20:46:49.599948 10717 ts_tablet_manager.cc:1367] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:49.600142 10716 tablet_bootstrap.cc:654] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:49.600404 10717 raft_consensus.cc:348] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:49.600508 10717 raft_consensus.cc:374] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:49.600548 10717 raft_consensus.cc:729] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Initialized, Role: FOLLOWER
I20240502 20:46:49.600670 10717 consensus_queue.cc:260] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:49.600921 10717 ts_tablet_manager.cc:1398] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:49.601015 10716 tablet_bootstrap.cc:492] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc: No bootstrap required, opened a new log
I20240502 20:46:49.601070 10717 tablet_bootstrap.cc:492] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448: Bootstrap starting.
I20240502 20:46:49.601101 10716 ts_tablet_manager.cc:1367] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:49.601454 10716 raft_consensus.cc:348] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:49.601552 10716 raft_consensus.cc:374] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:49.601583 10716 raft_consensus.cc:729] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Initialized, Role: FOLLOWER
I20240502 20:46:49.601671 10716 consensus_queue.cc:260] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:49.601922 10716 ts_tablet_manager.cc:1398] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:49.602066 10717 tablet_bootstrap.cc:654] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:49.602068 10716 tablet_bootstrap.cc:492] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc: Bootstrap starting.
I20240502 20:46:49.602820 10716 tablet_bootstrap.cc:654] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:49.603053 10717 tablet_bootstrap.cc:492] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448: No bootstrap required, opened a new log
I20240502 20:46:49.603132 10717 ts_tablet_manager.cc:1367] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:49.603533 10717 raft_consensus.cc:348] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:49.603610 10717 raft_consensus.cc:374] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:49.603641 10717 raft_consensus.cc:729] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Initialized, Role: FOLLOWER
I20240502 20:46:49.603766 10716 tablet_bootstrap.cc:492] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc: No bootstrap required, opened a new log
I20240502 20:46:49.603849 10716 ts_tablet_manager.cc:1367] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:49.603880 10717 consensus_queue.cc:260] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:49.604149 10717 ts_tablet_manager.cc:1398] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:49.604228 10716 raft_consensus.cc:348] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:49.604326 10716 raft_consensus.cc:374] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:49.604329 10717 tablet_bootstrap.cc:492] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448: Bootstrap starting.
I20240502 20:46:49.604363 10716 raft_consensus.cc:729] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Initialized, Role: FOLLOWER
I20240502 20:46:49.604485 10716 consensus_queue.cc:260] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:49.604748 10716 ts_tablet_manager.cc:1398] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:49.605278 10717 tablet_bootstrap.cc:654] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:49.606354 10717 tablet_bootstrap.cc:492] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448: No bootstrap required, opened a new log
I20240502 20:46:49.606434 10717 ts_tablet_manager.cc:1367] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:49.606868 10717 raft_consensus.cc:348] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:49.606962 10717 raft_consensus.cc:374] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:49.606997 10717 raft_consensus.cc:729] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Initialized, Role: FOLLOWER
I20240502 20:46:49.607129 10717 consensus_queue.cc:260] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:49.607383 10717 ts_tablet_manager.cc:1398] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:49.641398 10689 raft_consensus.cc:482] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:49.641499 10689 raft_consensus.cc:504] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:49.641810 10689 leader_election.cc:288] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:49.642105 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc" is_pre_election: true
I20240502 20:46:49.642117 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64136bd82a30486e9d56fc28da4bd448" is_pre_election: true
I20240502 20:46:49.642228 9688 raft_consensus.cc:2454] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 0.
I20240502 20:46:49.642235 9423 raft_consensus.cc:2454] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 0.
I20240502 20:46:49.642511 9223 leader_election.cc:302] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:49.642692 10689 raft_consensus.cc:2781] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:49.642745 10689 raft_consensus.cc:482] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:49.642789 10689 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:49.643565 10689 raft_consensus.cc:504] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:49.643857 10689 leader_election.cc:288] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 1 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:49.644140 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:49.644174 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:49.644229 9688 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:49.644268 9423 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:49.644884 9688 raft_consensus.cc:2454] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 1.
I20240502 20:46:49.644970 9423 raft_consensus.cc:2454] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 1.
I20240502 20:46:49.645129 9223 leader_election.cc:302] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:49.645265 10689 raft_consensus.cc:2781] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:49.645354 10689 raft_consensus.cc:686] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:49.645476 10689 consensus_queue.cc:237] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:49.646935 9136 catalog_manager.cc:5484] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 1 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:49.688234 10634 raft_consensus.cc:482] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:49.688344 10634 raft_consensus.cc:504] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:49.688752 10634 leader_election.cc:288] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:49.689097 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc" is_pre_election: true
I20240502 20:46:49.689113 9556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" is_pre_election: true
I20240502 20:46:49.689218 9423 raft_consensus.cc:2454] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 0.
I20240502 20:46:49.689231 9556 raft_consensus.cc:2454] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 0.
I20240502 20:46:49.689505 9622 leader_election.cc:302] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:49.689687 10634 raft_consensus.cc:2781] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:49.689740 10634 raft_consensus.cc:482] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:49.689777 10634 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:49.690541 10634 raft_consensus.cc:504] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:49.690833 10634 leader_election.cc:288] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:49.691136 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:49.691143 9556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:49.691231 9423 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:49.691239 9556 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:49.691887 9423 raft_consensus.cc:2454] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 1.
I20240502 20:46:49.692090 9556 raft_consensus.cc:2454] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 1.
I20240502 20:46:49.692153 9622 leader_election.cc:302] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:49.692303 10634 raft_consensus.cc:2781] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:49.692487 10634 raft_consensus.cc:686] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 1 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:49.692612 10634 consensus_queue.cc:237] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:49.694116 9142 catalog_manager.cc:5484] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:49.695181 10634 raft_consensus.cc:482] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:49.695276 10634 raft_consensus.cc:504] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:49.695884 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc" is_pre_election: true
I20240502 20:46:49.695923 10634 leader_election.cc:288] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:49.695982 9423 raft_consensus.cc:2454] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 0.
I20240502 20:46:49.696195 9288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" is_pre_election: true
I20240502 20:46:49.696301 9288 raft_consensus.cc:2454] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 0.
I20240502 20:46:49.696588 9621 leader_election.cc:302] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:49.696779 10634 raft_consensus.cc:2781] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:49.696822 10634 raft_consensus.cc:482] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:49.696846 10634 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:49.697547 10634 raft_consensus.cc:504] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:49.697844 10634 leader_election.cc:288] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:49.698158 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:49.698165 9288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:49.698233 9423 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:49.698266 9288 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:49.698868 9423 raft_consensus.cc:2454] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 1.
I20240502 20:46:49.699106 9622 leader_election.cc:302] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:49.699118 9288 raft_consensus.cc:2454] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 1.
I20240502 20:46:49.699237 10634 raft_consensus.cc:2781] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:49.699319 10634 raft_consensus.cc:686] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 1 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:49.699558 10634 consensus_queue.cc:237] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:49.700861 9142 catalog_manager.cc:5484] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
W20240502 20:46:49.826040 10728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:49.826248 10728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:49.829200 10728 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_6'
I20240502 20:46:49.921169 9688 raft_consensus.cc:1261] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:49.921576 9288 raft_consensus.cc:1261] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:49.921715 10689 consensus_queue.cc:1035] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:49.921833 9423 raft_consensus.cc:1261] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:49.922237 10689 consensus_queue.cc:1035] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:49.923131 10725 consensus_queue.cc:1035] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:49.925248 9423 raft_consensus.cc:1261] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:49.925693 10634 consensus_queue.cc:1035] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:49.925992 9424 raft_consensus.cc:1261] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:49.928432 10725 consensus_queue.cc:1035] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:49.930444 9556 raft_consensus.cc:1261] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:49.931067 10725 consensus_queue.cc:1035] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:49.943621 10690 mvcc.cc:205] Tried to move back new op lower bound from 7023340789434736640 to 7023340788530434048. Current Snapshot: MvccSnapshot[applied={T|T < 7023340789434736640}]
INSERT report
rows total: 2000
time total: 110.704 ms
time per row: 0.0553519 ms
I20240502 20:46:49.981109 9141 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33288:
name: "rebalance_test_table_7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:49.986006 9535 tablet_service.cc:1467] Processing CreateTablet for tablet a8b6c379ec40417894f65f3c782c28db (DEFAULT_TABLE table=rebalance_test_table_7 [id=30207bd5081f4593b4554fbdf80dd883]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:49.986078 9268 tablet_service.cc:1467] Processing CreateTablet for tablet a8b6c379ec40417894f65f3c782c28db (DEFAULT_TABLE table=rebalance_test_table_7 [id=30207bd5081f4593b4554fbdf80dd883]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:49.986300 9535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8b6c379ec40417894f65f3c782c28db. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:49.986336 9268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8b6c379ec40417894f65f3c782c28db. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:49.986670 9402 tablet_service.cc:1467] Processing CreateTablet for tablet a8b6c379ec40417894f65f3c782c28db (DEFAULT_TABLE table=rebalance_test_table_7 [id=30207bd5081f4593b4554fbdf80dd883]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:49.986912 9402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8b6c379ec40417894f65f3c782c28db. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:49.988001 10709 tablet_bootstrap.cc:492] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f: Bootstrap starting.
I20240502 20:46:49.988500 9268 tablet_service.cc:1467] Processing CreateTablet for tablet bf6aa5b2d4c04d2bb971dabc21a4544a (DEFAULT_TABLE table=rebalance_test_table_7 [id=30207bd5081f4593b4554fbdf80dd883]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:49.988586 10710 tablet_bootstrap.cc:492] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e: Bootstrap starting.
I20240502 20:46:49.988700 9268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf6aa5b2d4c04d2bb971dabc21a4544a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:49.988788 10709 tablet_bootstrap.cc:654] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:49.988893 10716 tablet_bootstrap.cc:492] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc: Bootstrap starting.
I20240502 20:46:49.989413 10710 tablet_bootstrap.cc:654] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:49.989676 10716 tablet_bootstrap.cc:654] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:49.990443 10710 tablet_bootstrap.cc:492] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e: No bootstrap required, opened a new log
I20240502 20:46:49.990561 10710 ts_tablet_manager.cc:1367] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:49.990532 9402 tablet_service.cc:1467] Processing CreateTablet for tablet 14d5186eb2e84ed19ab0f981067d3775 (DEFAULT_TABLE table=rebalance_test_table_7 [id=30207bd5081f4593b4554fbdf80dd883]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:49.990746 9402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14d5186eb2e84ed19ab0f981067d3775. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:49.990986 10710 raft_consensus.cc:348] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:49.991094 10710 raft_consensus.cc:374] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:49.991125 10710 raft_consensus.cc:729] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Initialized, Role: FOLLOWER
I20240502 20:46:49.991071 9669 tablet_service.cc:1467] Processing CreateTablet for tablet bf6aa5b2d4c04d2bb971dabc21a4544a (DEFAULT_TABLE table=rebalance_test_table_7 [id=30207bd5081f4593b4554fbdf80dd883]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:49.991147 10709 tablet_bootstrap.cc:492] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f: No bootstrap required, opened a new log
I20240502 20:46:49.991215 10709 ts_tablet_manager.cc:1367] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 20:46:49.991235 10710 consensus_queue.cc:260] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:49.991312 9669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf6aa5b2d4c04d2bb971dabc21a4544a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:49.991511 10710 ts_tablet_manager.cc:1398] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:49.991612 10709 raft_consensus.cc:348] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:49.991700 10709 raft_consensus.cc:374] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:49.991732 10709 raft_consensus.cc:729] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Initialized, Role: FOLLOWER
I20240502 20:46:49.991842 10709 consensus_queue.cc:260] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:49.992110 10709 ts_tablet_manager.cc:1398] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:49.992285 10709 tablet_bootstrap.cc:492] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f: Bootstrap starting.
I20240502 20:46:49.992832 9535 tablet_service.cc:1467] Processing CreateTablet for tablet bf6aa5b2d4c04d2bb971dabc21a4544a (DEFAULT_TABLE table=rebalance_test_table_7 [id=30207bd5081f4593b4554fbdf80dd883]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:49.992909 9536 tablet_service.cc:1467] Processing CreateTablet for tablet 14d5186eb2e84ed19ab0f981067d3775 (DEFAULT_TABLE table=rebalance_test_table_7 [id=30207bd5081f4593b4554fbdf80dd883]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:49.993053 9535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf6aa5b2d4c04d2bb971dabc21a4544a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:49.993216 10709 tablet_bootstrap.cc:654] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:49.993254 9536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14d5186eb2e84ed19ab0f981067d3775. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:49.993475 9667 tablet_service.cc:1467] Processing CreateTablet for tablet 14d5186eb2e84ed19ab0f981067d3775 (DEFAULT_TABLE table=rebalance_test_table_7 [id=30207bd5081f4593b4554fbdf80dd883]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:49.993670 9667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14d5186eb2e84ed19ab0f981067d3775. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:49.994577 10717 tablet_bootstrap.cc:492] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448: Bootstrap starting.
I20240502 20:46:49.995368 10717 tablet_bootstrap.cc:654] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:49.995373 10710 tablet_bootstrap.cc:492] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e: Bootstrap starting.
I20240502 20:46:49.996158 10710 tablet_bootstrap.cc:654] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:49.996441 10717 tablet_bootstrap.cc:492] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448: No bootstrap required, opened a new log
I20240502 20:46:49.996519 10717 ts_tablet_manager.cc:1367] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:49.996961 10717 raft_consensus.cc:348] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:49.997061 10717 raft_consensus.cc:374] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:49.997095 10717 raft_consensus.cc:729] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Initialized, Role: FOLLOWER
I20240502 20:46:49.997210 10717 consensus_queue.cc:260] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:49.997468 10717 ts_tablet_manager.cc:1398] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:49.997608 10717 tablet_bootstrap.cc:492] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448: Bootstrap starting.
I20240502 20:46:49.997642 10716 tablet_bootstrap.cc:492] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc: No bootstrap required, opened a new log
I20240502 20:46:49.997721 10716 ts_tablet_manager.cc:1367] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240502 20:46:49.998107 10716 raft_consensus.cc:348] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:49.998207 10716 raft_consensus.cc:374] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:49.998239 10716 raft_consensus.cc:729] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Initialized, Role: FOLLOWER
I20240502 20:46:49.998351 10716 consensus_queue.cc:260] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:49.998674 10716 ts_tablet_manager.cc:1398] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:49.998842 10716 tablet_bootstrap.cc:492] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc: Bootstrap starting.
I20240502 20:46:49.999589 10716 tablet_bootstrap.cc:654] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:49.999925 10717 tablet_bootstrap.cc:654] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:50.000960 10709 tablet_bootstrap.cc:492] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f: No bootstrap required, opened a new log
I20240502 20:46:50.001042 10709 ts_tablet_manager.cc:1367] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240502 20:46:50.001201 10717 tablet_bootstrap.cc:492] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448: No bootstrap required, opened a new log
I20240502 20:46:50.001272 10717 ts_tablet_manager.cc:1367] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 20:46:50.001461 10709 raft_consensus.cc:348] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:50.001554 10709 raft_consensus.cc:374] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:50.001582 10709 raft_consensus.cc:729] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Initialized, Role: FOLLOWER
I20240502 20:46:50.001655 10717 raft_consensus.cc:348] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:50.001724 10717 raft_consensus.cc:374] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:50.001688 10709 consensus_queue.cc:260] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:50.001742 10717 raft_consensus.cc:729] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Initialized, Role: FOLLOWER
I20240502 20:46:50.001830 10717 consensus_queue.cc:260] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:50.001927 10709 ts_tablet_manager.cc:1398] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:50.002094 10717 ts_tablet_manager.cc:1398] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:50.002318 10716 tablet_bootstrap.cc:492] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc: No bootstrap required, opened a new log
I20240502 20:46:50.002396 10716 ts_tablet_manager.cc:1367] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:50.002807 10716 raft_consensus.cc:348] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:50.002899 10716 raft_consensus.cc:374] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:50.002929 10716 raft_consensus.cc:729] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Initialized, Role: FOLLOWER
I20240502 20:46:50.003028 10716 consensus_queue.cc:260] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:50.003122 10710 tablet_bootstrap.cc:492] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e: No bootstrap required, opened a new log
I20240502 20:46:50.003186 10710 ts_tablet_manager.cc:1367] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20240502 20:46:50.003273 10716 ts_tablet_manager.cc:1398] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:50.003587 10710 raft_consensus.cc:348] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:50.003684 10710 raft_consensus.cc:374] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:50.003713 10710 raft_consensus.cc:729] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Initialized, Role: FOLLOWER
I20240502 20:46:50.003823 10710 consensus_queue.cc:260] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:50.004078 10710 ts_tablet_manager.cc:1398] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:50.004233 10710 tablet_bootstrap.cc:492] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e: Bootstrap starting.
I20240502 20:46:50.005021 10710 tablet_bootstrap.cc:654] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:50.005985 10710 tablet_bootstrap.cc:492] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e: No bootstrap required, opened a new log
I20240502 20:46:50.006058 10710 ts_tablet_manager.cc:1367] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:50.006434 10710 raft_consensus.cc:348] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:50.006572 10710 raft_consensus.cc:374] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:50.006605 10710 raft_consensus.cc:729] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Initialized, Role: FOLLOWER
I20240502 20:46:50.006752 10710 consensus_queue.cc:260] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:50.007040 10710 ts_tablet_manager.cc:1398] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:50.102551 10725 raft_consensus.cc:482] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:50.102656 10725 raft_consensus.cc:504] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:50.103041 10725 leader_election.cc:288] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:50.103367 9556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" is_pre_election: true
I20240502 20:46:50.103396 9288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" is_pre_election: true
I20240502 20:46:50.103508 9556 raft_consensus.cc:2454] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 0.
I20240502 20:46:50.103564 9288 raft_consensus.cc:2454] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 0.
I20240502 20:46:50.103776 9623 leader_election.cc:302] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:50.103952 10725 raft_consensus.cc:2781] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:50.104013 10725 raft_consensus.cc:482] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:50.104056 10725 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:50.104844 10725 raft_consensus.cc:504] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:50.105127 10725 leader_election.cc:288] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:50.105429 9556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:50.105430 9288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:50.105537 9288 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:50.105537 9556 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:50.106410 9288 raft_consensus.cc:2454] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 1.
I20240502 20:46:50.106420 9556 raft_consensus.cc:2454] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 1.
I20240502 20:46:50.106705 9623 leader_election.cc:302] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:50.106868 10725 raft_consensus.cc:2781] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:50.106956 10725 raft_consensus.cc:686] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 1 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:50.107079 10725 consensus_queue.cc:237] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:50.108577 9143 catalog_manager.cc:5484] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 1 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:50.196382 10462 raft_consensus.cc:482] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:50.196514 10462 raft_consensus.cc:504] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:50.196879 10462 leader_election.cc:288] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:50.197252 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64136bd82a30486e9d56fc28da4bd448" is_pre_election: true
I20240502 20:46:50.197295 9556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" is_pre_election: true
I20240502 20:46:50.197391 9688 raft_consensus.cc:2454] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 0.
I20240502 20:46:50.197415 9556 raft_consensus.cc:2454] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 0.
I20240502 20:46:50.197674 9357 leader_election.cc:302] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:50.197839 10462 raft_consensus.cc:2781] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:50.197906 10462 raft_consensus.cc:482] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:50.197942 10462 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:50.198750 10462 raft_consensus.cc:504] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:50.199048 10462 leader_election.cc:288] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:50.199352 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:50.199409 9556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:50.199460 9688 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:50.199527 9556 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:50.200196 9688 raft_consensus.cc:2454] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 1.
I20240502 20:46:50.200238 9556 raft_consensus.cc:2454] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 1.
I20240502 20:46:50.200449 9357 leader_election.cc:302] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:50.200621 10462 raft_consensus.cc:2781] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:50.200703 10462 raft_consensus.cc:686] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:50.200850 10462 consensus_queue.cc:237] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:50.202420 9143 catalog_manager.cc:5484] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 1 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:50.243875 10636 raft_consensus.cc:482] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:50.243965 10636 raft_consensus.cc:504] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.244282 10636 leader_election.cc:288] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:50.244587 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc" is_pre_election: true
I20240502 20:46:50.244604 9288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" is_pre_election: true
I20240502 20:46:50.244704 9288 raft_consensus.cc:2454] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 0.
I20240502 20:46:50.244714 9424 raft_consensus.cc:2454] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 0.
I20240502 20:46:50.244951 9490 leader_election.cc:302] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:50.245110 10636 raft_consensus.cc:2781] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:50.245170 10636 raft_consensus.cc:482] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:50.245203 10636 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:50.245882 10636 raft_consensus.cc:504] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.246153 10636 leader_election.cc:288] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 1 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:50.246474 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:50.246526 9288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:50.246580 9424 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:50.246630 9288 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:50.247457 9424 raft_consensus.cc:2454] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 1.
I20240502 20:46:50.247457 9288 raft_consensus.cc:2454] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 1.
I20240502 20:46:50.247725 9490 leader_election.cc:302] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:50.247889 10636 raft_consensus.cc:2781] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:50.247972 10636 raft_consensus.cc:686] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:50.248098 10636 consensus_queue.cc:237] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.249605 9143 catalog_manager.cc:5484] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 1 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
W20240502 20:46:50.386088 10782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:50.386286 10782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:50.389133 10782 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_7'
I20240502 20:46:50.488349 10725 consensus_queue.cc:1035] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:50.489531 9287 raft_consensus.cc:1261] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:50.489939 9424 raft_consensus.cc:1261] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:50.490532 10554 consensus_queue.cc:1035] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:50.490664 10636 consensus_queue.cc:1035] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:50.491448 9556 raft_consensus.cc:1261] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:50.491834 10725 consensus_queue.cc:1035] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:50.509227 9556 raft_consensus.cc:1261] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:50.509583 10462 consensus_queue.cc:1035] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:50.513163 9688 raft_consensus.cc:1261] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:50.513901 10774 consensus_queue.cc:1035] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 136.17 ms
time per row: 0.0680852 ms
I20240502 20:46:50.565212 9144 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33288:
name: "rebalance_test_table_8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:50.570145 9402 tablet_service.cc:1467] Processing CreateTablet for tablet f6edce87fbb8469a94d38d6d2eab622c (DEFAULT_TABLE table=rebalance_test_table_8 [id=31b237bc3b89483486aa6053289581e1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:50.570349 9536 tablet_service.cc:1467] Processing CreateTablet for tablet f6edce87fbb8469a94d38d6d2eab622c (DEFAULT_TABLE table=rebalance_test_table_8 [id=31b237bc3b89483486aa6053289581e1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:50.570420 9667 tablet_service.cc:1467] Processing CreateTablet for tablet 2cf930db9da24ba5b1f2df86d025d376 (DEFAULT_TABLE table=rebalance_test_table_8 [id=31b237bc3b89483486aa6053289581e1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:50.570511 9402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6edce87fbb8469a94d38d6d2eab622c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:50.570639 9536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6edce87fbb8469a94d38d6d2eab622c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:50.570674 9667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cf930db9da24ba5b1f2df86d025d376. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:50.571233 9268 tablet_service.cc:1467] Processing CreateTablet for tablet f6edce87fbb8469a94d38d6d2eab622c (DEFAULT_TABLE table=rebalance_test_table_8 [id=31b237bc3b89483486aa6053289581e1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:50.571475 9268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6edce87fbb8469a94d38d6d2eab622c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:50.572795 9269 tablet_service.cc:1467] Processing CreateTablet for tablet 2cf930db9da24ba5b1f2df86d025d376 (DEFAULT_TABLE table=rebalance_test_table_8 [id=31b237bc3b89483486aa6053289581e1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:50.572980 10847 tablet_bootstrap.cc:492] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448: Bootstrap starting.
I20240502 20:46:50.572996 9269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cf930db9da24ba5b1f2df86d025d376. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:50.573773 10847 tablet_bootstrap.cc:654] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:50.574235 9536 tablet_service.cc:1467] Processing CreateTablet for tablet 6278a8937308475d88b2b53e634dcdd5 (DEFAULT_TABLE table=rebalance_test_table_8 [id=31b237bc3b89483486aa6053289581e1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:50.574395 10846 tablet_bootstrap.cc:492] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e: Bootstrap starting.
I20240502 20:46:50.574371 9667 tablet_service.cc:1467] Processing CreateTablet for tablet 6278a8937308475d88b2b53e634dcdd5 (DEFAULT_TABLE table=rebalance_test_table_8 [id=31b237bc3b89483486aa6053289581e1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:50.574452 9536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6278a8937308475d88b2b53e634dcdd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:50.574605 9667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6278a8937308475d88b2b53e634dcdd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:50.574694 9267 tablet_service.cc:1467] Processing CreateTablet for tablet 6278a8937308475d88b2b53e634dcdd5 (DEFAULT_TABLE table=rebalance_test_table_8 [id=31b237bc3b89483486aa6053289581e1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:50.574900 9267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6278a8937308475d88b2b53e634dcdd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:50.575393 9402 tablet_service.cc:1467] Processing CreateTablet for tablet 2cf930db9da24ba5b1f2df86d025d376 (DEFAULT_TABLE table=rebalance_test_table_8 [id=31b237bc3b89483486aa6053289581e1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:50.575609 9402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cf930db9da24ba5b1f2df86d025d376. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:50.577522 10845 tablet_bootstrap.cc:492] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc: Bootstrap starting.
I20240502 20:46:50.578269 10846 tablet_bootstrap.cc:654] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:50.578403 10845 tablet_bootstrap.cc:654] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:50.578940 10848 tablet_bootstrap.cc:492] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f: Bootstrap starting.
I20240502 20:46:50.579452 10845 tablet_bootstrap.cc:492] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc: No bootstrap required, opened a new log
I20240502 20:46:50.579493 10847 tablet_bootstrap.cc:492] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448: No bootstrap required, opened a new log
I20240502 20:46:50.579536 10845 ts_tablet_manager.cc:1367] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:50.579557 10847 ts_tablet_manager.cc:1367] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 20:46:50.579960 10847 raft_consensus.cc:348] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.579960 10845 raft_consensus.cc:348] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.580070 10845 raft_consensus.cc:374] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:50.580070 10847 raft_consensus.cc:374] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:50.580101 10847 raft_consensus.cc:729] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Initialized, Role: FOLLOWER
I20240502 20:46:50.580101 10845 raft_consensus.cc:729] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Initialized, Role: FOLLOWER
I20240502 20:46:50.580221 10847 consensus_queue.cc:260] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.580221 10845 consensus_queue.cc:260] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.580500 10845 ts_tablet_manager.cc:1398] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:50.580500 10847 ts_tablet_manager.cc:1398] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:50.580721 10847 tablet_bootstrap.cc:492] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448: Bootstrap starting.
I20240502 20:46:50.580909 10846 tablet_bootstrap.cc:492] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e: No bootstrap required, opened a new log
I20240502 20:46:50.580996 10846 ts_tablet_manager.cc:1367] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 20:46:50.581400 10846 raft_consensus.cc:348] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.581496 10846 raft_consensus.cc:374] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:50.581523 10846 raft_consensus.cc:729] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Initialized, Role: FOLLOWER
I20240502 20:46:50.581537 10847 tablet_bootstrap.cc:654] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:50.581670 10846 consensus_queue.cc:260] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.581964 10846 ts_tablet_manager.cc:1398] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:50.582127 10846 tablet_bootstrap.cc:492] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e: Bootstrap starting.
I20240502 20:46:50.582620 10845 tablet_bootstrap.cc:492] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc: Bootstrap starting.
I20240502 20:46:50.583847 10846 tablet_bootstrap.cc:654] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:50.584139 10845 tablet_bootstrap.cc:654] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:50.584688 10847 tablet_bootstrap.cc:492] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448: No bootstrap required, opened a new log
I20240502 20:46:50.584843 10848 tablet_bootstrap.cc:654] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:50.584821 10847 ts_tablet_manager.cc:1367] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 20:46:50.585539 10847 raft_consensus.cc:348] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.585644 10848 tablet_bootstrap.cc:492] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f: No bootstrap required, opened a new log
I20240502 20:46:50.585721 10847 raft_consensus.cc:374] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:50.585734 10848 ts_tablet_manager.cc:1367] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 20:46:50.585821 10847 raft_consensus.cc:729] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Initialized, Role: FOLLOWER
I20240502 20:46:50.586005 10847 consensus_queue.cc:260] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.586077 10848 raft_consensus.cc:348] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.586161 10848 raft_consensus.cc:374] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:50.586180 10848 raft_consensus.cc:729] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Initialized, Role: FOLLOWER
I20240502 20:46:50.586273 10848 consensus_queue.cc:260] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.586547 10848 ts_tablet_manager.cc:1398] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:50.586694 10848 tablet_bootstrap.cc:492] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f: Bootstrap starting.
I20240502 20:46:50.586937 10847 ts_tablet_manager.cc:1398] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:50.587347 10848 tablet_bootstrap.cc:654] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:50.588402 10846 tablet_bootstrap.cc:492] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e: No bootstrap required, opened a new log
I20240502 20:46:50.588479 10846 ts_tablet_manager.cc:1367] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 20:46:50.588836 10846 raft_consensus.cc:348] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.588925 10846 raft_consensus.cc:374] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:50.588953 10846 raft_consensus.cc:729] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Initialized, Role: FOLLOWER
I20240502 20:46:50.589059 10846 consensus_queue.cc:260] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.589299 10846 ts_tablet_manager.cc:1398] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:50.589725 10848 tablet_bootstrap.cc:492] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f: No bootstrap required, opened a new log
I20240502 20:46:50.589814 10848 ts_tablet_manager.cc:1367] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:50.590167 10848 raft_consensus.cc:348] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.590247 10848 raft_consensus.cc:374] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:50.590269 10848 raft_consensus.cc:729] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Initialized, Role: FOLLOWER
I20240502 20:46:50.590368 10848 consensus_queue.cc:260] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.590639 10848 ts_tablet_manager.cc:1398] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:50.590790 10848 tablet_bootstrap.cc:492] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f: Bootstrap starting.
I20240502 20:46:50.591550 10848 tablet_bootstrap.cc:654] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:50.592523 10848 tablet_bootstrap.cc:492] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f: No bootstrap required, opened a new log
I20240502 20:46:50.592607 10848 ts_tablet_manager.cc:1367] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 20:46:50.592954 10848 raft_consensus.cc:348] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.593055 10848 raft_consensus.cc:374] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:50.593096 10848 raft_consensus.cc:729] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Initialized, Role: FOLLOWER
I20240502 20:46:50.593211 10848 consensus_queue.cc:260] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.593540 10848 ts_tablet_manager.cc:1398] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 20:46:50.594020 10845 tablet_bootstrap.cc:492] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc: No bootstrap required, opened a new log
I20240502 20:46:50.594098 10845 ts_tablet_manager.cc:1367] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.002s
I20240502 20:46:50.594489 10845 raft_consensus.cc:348] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.594583 10845 raft_consensus.cc:374] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:50.594612 10845 raft_consensus.cc:729] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Initialized, Role: FOLLOWER
I20240502 20:46:50.594738 10845 consensus_queue.cc:260] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.594993 10845 ts_tablet_manager.cc:1398] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:50.688225 10838 raft_consensus.cc:482] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:50.688333 10838 raft_consensus.cc:504] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.688696 10838 leader_election.cc:288] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:50.689061 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" is_pre_election: true
I20240502 20:46:50.689042 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64136bd82a30486e9d56fc28da4bd448" is_pre_election: true
I20240502 20:46:50.689188 9688 raft_consensus.cc:2454] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 0.
I20240502 20:46:50.689188 9287 raft_consensus.cc:2454] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 0.
I20240502 20:46:50.689451 9355 leader_election.cc:302] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:50.689635 10838 raft_consensus.cc:2781] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:50.689702 10838 raft_consensus.cc:482] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:50.689733 10838 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:50.690716 10838 raft_consensus.cc:504] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.691052 10838 leader_election.cc:288] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:50.691323 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:50.691331 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:50.691431 9688 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:50.691435 9287 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:50.692272 9688 raft_consensus.cc:2454] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 1.
I20240502 20:46:50.692416 9287 raft_consensus.cc:2454] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 1.
I20240502 20:46:50.692518 9357 leader_election.cc:302] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:50.692721 10838 raft_consensus.cc:2781] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:50.692808 10838 raft_consensus.cc:686] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:50.692937 10838 consensus_queue.cc:237] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.694758 9140 catalog_manager.cc:5484] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 1 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:50.698215 10820 raft_consensus.cc:482] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:50.698590 10820 raft_consensus.cc:504] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.698993 10820 leader_election.cc:288] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:50.699283 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64136bd82a30486e9d56fc28da4bd448" is_pre_election: true
I20240502 20:46:50.699386 9688 raft_consensus.cc:2454] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 0.
I20240502 20:46:50.699460 9556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" is_pre_election: true
I20240502 20:46:50.699587 9556 raft_consensus.cc:2454] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 0.
I20240502 20:46:50.699668 9223 leader_election.cc:302] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:50.699807 10820 raft_consensus.cc:2781] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:50.699860 10820 raft_consensus.cc:482] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:50.699900 10820 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:50.700605 10820 raft_consensus.cc:504] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.700881 10820 leader_election.cc:288] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 1 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:50.701191 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:50.701210 9556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:50.701287 9688 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:50.701316 9556 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:50.701962 9556 raft_consensus.cc:2454] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 1.
I20240502 20:46:50.702064 9688 raft_consensus.cc:2454] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 1.
I20240502 20:46:50.702242 9223 leader_election.cc:302] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:50.702414 10820 raft_consensus.cc:2781] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:50.702513 10820 raft_consensus.cc:686] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:50.702639 10820 consensus_queue.cc:237] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.704159 9140 catalog_manager.cc:5484] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 1 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:50.716897 10774 raft_consensus.cc:482] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:50.716975 10774 raft_consensus.cc:504] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.717288 10774 leader_election.cc:288] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:50.717584 9556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" is_pre_election: true
I20240502 20:46:50.717582 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" is_pre_election: true
I20240502 20:46:50.717675 9287 raft_consensus.cc:2454] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 0.
I20240502 20:46:50.717684 9556 raft_consensus.cc:2454] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 0.
I20240502 20:46:50.717912 9357 leader_election.cc:302] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:50.718053 10774 raft_consensus.cc:2781] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:50.718109 10774 raft_consensus.cc:482] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:50.718138 10774 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:50.718917 10774 raft_consensus.cc:504] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.719200 10774 leader_election.cc:288] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:50.719456 9556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:50.719491 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:50.719548 9556 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:50.719586 9287 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:50.720198 9287 raft_consensus.cc:2454] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 1.
I20240502 20:46:50.720203 9556 raft_consensus.cc:2454] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 1.
I20240502 20:46:50.720444 9357 leader_election.cc:302] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:50.720592 10774 raft_consensus.cc:2781] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:50.720675 10774 raft_consensus.cc:686] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 1 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:50.720791 10774 consensus_queue.cc:237] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:50.722213 9140 catalog_manager.cc:5484] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 1 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
W20240502 20:46:50.830049 10861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:50.830240 10861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:50.833369 10861 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_8'
I20240502 20:46:50.903848 9688 raft_consensus.cc:1261] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:50.904820 9556 raft_consensus.cc:1261] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:50.904896 9287 raft_consensus.cc:1261] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:50.905105 10774 consensus_queue.cc:1035] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:50.905392 10774 consensus_queue.cc:1035] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:50.905889 10838 consensus_queue.cc:1035] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:50.911057 9556 raft_consensus.cc:1261] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:50.911434 10689 consensus_queue.cc:1035] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:50.913013 9688 raft_consensus.cc:1261] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:50.915516 9287 raft_consensus.cc:1261] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:50.917898 10689 consensus_queue.cc:1035] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:50.922817 10462 consensus_queue.cc:1035] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 102.129 ms
time per row: 0.0510646 ms
I20240502 20:46:50.974229 9140 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33288:
name: "rebalance_test_table_9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 20:46:50.979254 9536 tablet_service.cc:1467] Processing CreateTablet for tablet c84bf8e0914f4ed4806e1e52d37c891a (DEFAULT_TABLE table=rebalance_test_table_9 [id=2458c2cd211c41e191ae0974391ba5db]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:50.979557 9536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c84bf8e0914f4ed4806e1e52d37c891a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:50.979591 9402 tablet_service.cc:1467] Processing CreateTablet for tablet c84bf8e0914f4ed4806e1e52d37c891a (DEFAULT_TABLE table=rebalance_test_table_9 [id=2458c2cd211c41e191ae0974391ba5db]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:50.979847 9402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c84bf8e0914f4ed4806e1e52d37c891a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:50.979902 9667 tablet_service.cc:1467] Processing CreateTablet for tablet c84bf8e0914f4ed4806e1e52d37c891a (DEFAULT_TABLE table=rebalance_test_table_9 [id=2458c2cd211c41e191ae0974391ba5db]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:50.980155 9667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c84bf8e0914f4ed4806e1e52d37c891a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:50.981846 10846 tablet_bootstrap.cc:492] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e: Bootstrap starting.
I20240502 20:46:50.981925 10847 tablet_bootstrap.cc:492] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448: Bootstrap starting.
I20240502 20:46:50.982110 9269 tablet_service.cc:1467] Processing CreateTablet for tablet 61b8dfbd38ea40ddb4b5c8f2b39aa43c (DEFAULT_TABLE table=rebalance_test_table_9 [id=2458c2cd211c41e191ae0974391ba5db]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:50.982362 9269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61b8dfbd38ea40ddb4b5c8f2b39aa43c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:50.982582 10845 tablet_bootstrap.cc:492] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc: Bootstrap starting.
I20240502 20:46:50.982733 10846 tablet_bootstrap.cc:654] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:50.983366 10845 tablet_bootstrap.cc:654] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:50.983839 10846 tablet_bootstrap.cc:492] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e: No bootstrap required, opened a new log
I20240502 20:46:50.983928 10846 ts_tablet_manager.cc:1367] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:50.984402 10848 tablet_bootstrap.cc:492] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f: Bootstrap starting.
I20240502 20:46:50.984355 10846 raft_consensus.cc:348] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:50.984457 10846 raft_consensus.cc:374] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:50.984485 10846 raft_consensus.cc:729] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Initialized, Role: FOLLOWER
I20240502 20:46:50.984560 9535 tablet_service.cc:1467] Processing CreateTablet for tablet 61b8dfbd38ea40ddb4b5c8f2b39aa43c (DEFAULT_TABLE table=rebalance_test_table_9 [id=2458c2cd211c41e191ae0974391ba5db]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:50.984594 10846 consensus_queue.cc:260] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:50.984782 9535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61b8dfbd38ea40ddb4b5c8f2b39aa43c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:50.984851 10846 ts_tablet_manager.cc:1398] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:50.985093 10847 tablet_bootstrap.cc:654] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:50.985191 10848 tablet_bootstrap.cc:654] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:50.985221 9536 tablet_service.cc:1467] Processing CreateTablet for tablet d4e11672a55842aa92ffb09edfc7546e (DEFAULT_TABLE table=rebalance_test_table_9 [id=2458c2cd211c41e191ae0974391ba5db]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:50.985432 9536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4e11672a55842aa92ffb09edfc7546e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:50.986235 10845 tablet_bootstrap.cc:492] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc: No bootstrap required, opened a new log
I20240502 20:46:50.986307 10845 ts_tablet_manager.cc:1367] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 20:46:50.986322 10848 tablet_bootstrap.cc:492] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f: No bootstrap required, opened a new log
I20240502 20:46:50.986403 10848 ts_tablet_manager.cc:1367] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 20:46:50.986835 10848 raft_consensus.cc:348] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:50.986929 10848 raft_consensus.cc:374] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:50.986959 10848 raft_consensus.cc:729] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Initialized, Role: FOLLOWER
I20240502 20:46:50.987063 10848 consensus_queue.cc:260] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:50.987308 10848 ts_tablet_manager.cc:1398] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:50.987404 10845 raft_consensus.cc:348] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:50.987514 10845 raft_consensus.cc:374] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:50.987543 10845 raft_consensus.cc:729] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Initialized, Role: FOLLOWER
I20240502 20:46:50.987637 9402 tablet_service.cc:1467] Processing CreateTablet for tablet d4e11672a55842aa92ffb09edfc7546e (DEFAULT_TABLE table=rebalance_test_table_9 [id=2458c2cd211c41e191ae0974391ba5db]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:50.987843 9402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4e11672a55842aa92ffb09edfc7546e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:50.987987 10847 tablet_bootstrap.cc:492] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448: No bootstrap required, opened a new log
I20240502 20:46:50.988044 10847 ts_tablet_manager.cc:1367] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 20:46:50.988373 10847 raft_consensus.cc:348] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:50.988467 10847 raft_consensus.cc:374] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:50.988495 10847 raft_consensus.cc:729] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Initialized, Role: FOLLOWER
I20240502 20:46:50.988610 10847 consensus_queue.cc:260] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:50.988715 9667 tablet_service.cc:1467] Processing CreateTablet for tablet d4e11672a55842aa92ffb09edfc7546e (DEFAULT_TABLE table=rebalance_test_table_9 [id=2458c2cd211c41e191ae0974391ba5db]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:50.988842 10847 ts_tablet_manager.cc:1398] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:50.988917 9667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4e11672a55842aa92ffb09edfc7546e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:50.989771 9404 tablet_service.cc:1467] Processing CreateTablet for tablet 61b8dfbd38ea40ddb4b5c8f2b39aa43c (DEFAULT_TABLE table=rebalance_test_table_9 [id=2458c2cd211c41e191ae0974391ba5db]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 20:46:50.989981 9404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61b8dfbd38ea40ddb4b5c8f2b39aa43c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 20:46:50.987651 10845 consensus_queue.cc:260] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:50.991745 10845 ts_tablet_manager.cc:1398] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 20:46:50.991897 10845 tablet_bootstrap.cc:492] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc: Bootstrap starting.
I20240502 20:46:50.992368 10846 tablet_bootstrap.cc:492] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e: Bootstrap starting.
I20240502 20:46:50.992669 10845 tablet_bootstrap.cc:654] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:50.993152 10846 tablet_bootstrap.cc:654] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:50.994277 10846 tablet_bootstrap.cc:492] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e: No bootstrap required, opened a new log
I20240502 20:46:50.994362 10846 ts_tablet_manager.cc:1367] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:50.994798 10846 raft_consensus.cc:348] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:50.994897 10846 raft_consensus.cc:374] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:50.994984 10846 raft_consensus.cc:729] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Initialized, Role: FOLLOWER
I20240502 20:46:50.995128 10846 consensus_queue.cc:260] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:50.995409 10846 ts_tablet_manager.cc:1398] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:50.995674 10846 tablet_bootstrap.cc:492] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e: Bootstrap starting.
I20240502 20:46:50.996605 10846 tablet_bootstrap.cc:654] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:50.997236 10845 tablet_bootstrap.cc:492] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc: No bootstrap required, opened a new log
I20240502 20:46:50.997328 10845 ts_tablet_manager.cc:1367] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 20:46:50.997681 10845 raft_consensus.cc:348] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:50.997762 10845 raft_consensus.cc:374] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:50.997768 10846 tablet_bootstrap.cc:492] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e: No bootstrap required, opened a new log
I20240502 20:46:50.997792 10845 raft_consensus.cc:729] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Initialized, Role: FOLLOWER
I20240502 20:46:50.997850 10846 ts_tablet_manager.cc:1367] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:50.998261 10847 tablet_bootstrap.cc:492] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448: Bootstrap starting.
I20240502 20:46:50.998416 10846 raft_consensus.cc:348] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:50.998540 10846 raft_consensus.cc:374] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:50.998581 10846 raft_consensus.cc:729] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Initialized, Role: FOLLOWER
I20240502 20:46:50.998569 10845 consensus_queue.cc:260] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:50.998693 10846 consensus_queue.cc:260] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:50.998836 10845 ts_tablet_manager.cc:1398] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:50.998932 10846 ts_tablet_manager.cc:1398] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:50.999053 10847 tablet_bootstrap.cc:654] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:50.999431 10845 tablet_bootstrap.cc:492] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc: Bootstrap starting.
I20240502 20:46:51.000169 10847 tablet_bootstrap.cc:492] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448: No bootstrap required, opened a new log
I20240502 20:46:51.000241 10847 ts_tablet_manager.cc:1367] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:51.000243 10845 tablet_bootstrap.cc:654] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc: Neither blocks nor log segments found. Creating new log.
I20240502 20:46:51.000633 10847 raft_consensus.cc:348] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:51.000736 10847 raft_consensus.cc:374] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:51.000773 10847 raft_consensus.cc:729] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Initialized, Role: FOLLOWER
I20240502 20:46:51.000881 10847 consensus_queue.cc:260] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:51.001127 10847 ts_tablet_manager.cc:1398] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 20:46:51.002166 10845 tablet_bootstrap.cc:492] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc: No bootstrap required, opened a new log
I20240502 20:46:51.002283 10845 ts_tablet_manager.cc:1367] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 20:46:51.002713 10845 raft_consensus.cc:348] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:51.002828 10845 raft_consensus.cc:374] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 20:46:51.002869 10845 raft_consensus.cc:729] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Initialized, Role: FOLLOWER
I20240502 20:46:51.002987 10845 consensus_queue.cc:260] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:51.003247 10845 ts_tablet_manager.cc:1398] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 20:46:51.195973 10462 raft_consensus.cc:482] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:51.196080 10462 raft_consensus.cc:504] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:51.196398 10462 leader_election.cc:288] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:51.196707 9557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" is_pre_election: true
I20240502 20:46:51.196748 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64136bd82a30486e9d56fc28da4bd448" is_pre_election: true
I20240502 20:46:51.196835 9557 raft_consensus.cc:2454] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 0.
I20240502 20:46:51.196879 9688 raft_consensus.cc:2454] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 0.
I20240502 20:46:51.197094 9357 leader_election.cc:302] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:51.197299 10462 raft_consensus.cc:2781] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:51.197360 10462 raft_consensus.cc:482] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:51.197397 10462 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:51.198145 10462 raft_consensus.cc:504] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:51.198488 10462 leader_election.cc:288] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:51.198776 9557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:51.198868 9557 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:51.198837 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:51.198948 9688 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:51.199555 9557 raft_consensus.cc:2454] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 1.
I20240502 20:46:51.199661 9688 raft_consensus.cc:2454] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 1.
I20240502 20:46:51.199916 9357 leader_election.cc:302] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:51.200062 10462 raft_consensus.cc:2781] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:51.200143 10462 raft_consensus.cc:686] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 1 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:51.200280 10462 consensus_queue.cc:237] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:51.201838 9144 catalog_manager.cc:5484] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 1 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:51.226038 10554 raft_consensus.cc:482] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:51.226130 10554 raft_consensus.cc:504] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:51.226450 10554 leader_election.cc:288] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:51.226766 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64136bd82a30486e9d56fc28da4bd448" is_pre_election: true
I20240502 20:46:51.226761 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc" is_pre_election: true
I20240502 20:46:51.226867 9688 raft_consensus.cc:2454] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 0.
I20240502 20:46:51.226873 9424 raft_consensus.cc:2454] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 0.
I20240502 20:46:51.227113 9492 leader_election.cc:302] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:51.227280 10554 raft_consensus.cc:2781] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:51.227355 10554 raft_consensus.cc:482] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:51.227383 10554 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:51.228062 10554 raft_consensus.cc:504] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:51.228368 10554 leader_election.cc:288] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 1 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:51.228674 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:51.228675 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:51.228780 9424 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:51.228780 9688 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:51.229629 9424 raft_consensus.cc:2454] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 1.
I20240502 20:46:51.229629 9688 raft_consensus.cc:2454] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 1.
I20240502 20:46:51.229892 9492 leader_election.cc:302] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:51.230036 10554 raft_consensus.cc:2781] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:51.230118 10554 raft_consensus.cc:686] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:51.230237 10554 consensus_queue.cc:237] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:51.231676 9144 catalog_manager.cc:5484] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 1 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:51.258100 10462 raft_consensus.cc:482] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 20:46:51.258185 10462 raft_consensus.cc:504] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:51.258551 10462 leader_election.cc:288] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:51.258829 9557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" is_pre_election: true
I20240502 20:46:51.258839 9288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" is_pre_election: true
I20240502 20:46:51.258931 9557 raft_consensus.cc:2454] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 0.
I20240502 20:46:51.258961 9288 raft_consensus.cc:2454] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 0.
I20240502 20:46:51.259188 9357 leader_election.cc:302] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:51.259346 10462 raft_consensus.cc:2781] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 20:46:51.259403 10462 raft_consensus.cc:482] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 20:46:51.259429 10462 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:51.260087 10462 raft_consensus.cc:504] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:51.260426 10462 leader_election.cc:288] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:51.260691 9288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:51.260725 9557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:51.260784 9288 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:51.260807 9557 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 0 FOLLOWER]: Advancing to term 1
I20240502 20:46:51.261451 9557 raft_consensus.cc:2454] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 1.
I20240502 20:46:51.261494 9288 raft_consensus.cc:2454] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 1.
I20240502 20:46:51.261723 9357 leader_election.cc:302] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:51.261883 10462 raft_consensus.cc:2781] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Leader election won for term 1
I20240502 20:46:51.261965 10462 raft_consensus.cc:686] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 1 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:51.262082 10462 consensus_queue.cc:237] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:51.263602 9144 catalog_manager.cc:5484] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 1 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
W20240502 20:46:51.375769 10911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:51.375986 10911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:51.379113 10911 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_9'
I20240502 20:46:51.462745 9288 raft_consensus.cc:1261] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:51.463742 9557 raft_consensus.cc:1261] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:51.464021 9556 raft_consensus.cc:1261] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:51.464323 9688 raft_consensus.cc:1261] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:51.464982 10462 consensus_queue.cc:1035] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:51.465267 10462 consensus_queue.cc:1035] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:51.466627 9424 raft_consensus.cc:1261] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:51.466653 10838 consensus_queue.cc:1035] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:51.466943 10838 consensus_queue.cc:1035] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:51.467083 10554 consensus_queue.cc:1035] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:51.468508 9689 raft_consensus.cc:1261] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 20:46:51.468884 10554 consensus_queue.cc:1035] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 20:46:51.494697 10955 mvcc.cc:205] Tried to move back new op lower bound from 7023340795763425280 to 7023340794677452800. Current Snapshot: MvccSnapshot[applied={T|T < 7023340795749941248}]
INSERT report
rows total: 2000
time total: 114.129 ms
time per row: 0.0570646 ms
I20240502 20:46:51.531342 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.69:41807
--local_ip_for_outbound_sockets=127.22.45.69
--tserver_master_addrs=127.22.45.126:41377
--webserver_port=44027
--webserver_interface=127.22.45.69
--builtin_ntp_servers=127.22.45.84:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 20:46:51.634616 10968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:51.634796 10968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:46:51.634837 10968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:51.637745 10968 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:46:51.637800 10968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:46:51.637867 10968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.69
I20240502 20:46:51.641463 10968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.69:41807
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.22.45.69
--webserver_port=44027
--tserver_master_addrs=127.22.45.126:41377
--never_fsync=true
--heap_profile_path=/tmp/kudu.10968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.69
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:46:51.642143 10968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:46:51.643123 10968 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20240502 20:46:51.651182 10968 server_base.cc:1034] running on GCE node
W20240502 20:46:51.651549 10973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:51.651820 10974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:51.652206 10976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:46:51.652627 10968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:46:51.653198 10968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:46:51.654367 10968 hybrid_clock.cc:648] HybridClock initialized: now 1714682811654353 us; error 25 us; skew 500 ppm
I20240502 20:46:51.656294 10968 webserver.cc:460] Webserver started at http://127.22.45.69:44027/ using document root <none> and password file <none>
I20240502 20:46:51.656895 10968 fs_manager.cc:359] Metadata directory not provided
I20240502 20:46:51.656975 10968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:46:51.660099 10968 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20240502 20:46:51.661864 10982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:51.662796 10968 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240502 20:46:51.662891 10968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "af8c170629d044fead8b1e8f948cbeee"
format_stamp: "Formatted at 2024-05-02 20:46:41 on dist-test-slave-g7g5"
I20240502 20:46:51.663260 10968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:46:51.688688 10968 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:46:51.689172 10968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:46:51.689349 10968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:46:51.689849 10968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:46:51.690704 10968 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:46:51.690757 10968 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:51.690816 10968 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:46:51.690843 10968 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:51.701176 10968 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.69:41807
I20240502 20:46:51.701221 11097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.69:41807 every 8 connection(s)
I20240502 20:46:51.702119 10968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 20:46:51.707429 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 10968
I20240502 20:46:51.707866 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.70:37255
--local_ip_for_outbound_sockets=127.22.45.70
--tserver_master_addrs=127.22.45.126:41377
--webserver_port=35361
--webserver_interface=127.22.45.70
--builtin_ntp_servers=127.22.45.84:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:46:51.710680 11098 heartbeater.cc:347] Connected to a master server at 127.22.45.126:41377
I20240502 20:46:51.710870 11098 heartbeater.cc:464] Registering TS with master...
I20240502 20:46:51.711243 11098 heartbeater.cc:511] Master 127.22.45.126:41377 requested a full tablet report, sending...
I20240502 20:46:51.712337 9144 ts_manager.cc:194] Re-registered known tserver with Master: af8c170629d044fead8b1e8f948cbeee (127.22.45.69:41807)
I20240502 20:46:51.713153 9144 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.69:58855
W20240502 20:46:51.814152 11101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:51.814328 11101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:46:51.814368 11101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:51.817239 11101 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:46:51.817294 11101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:46:51.817356 11101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.70
I20240502 20:46:51.820748 11101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.70:37255
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.22.45.70
--webserver_port=35361
--tserver_master_addrs=127.22.45.126:41377
--never_fsync=true
--heap_profile_path=/tmp/kudu.11101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.70
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:46:51.821440 11101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:46:51.822366 11101 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:46:51.829742 11106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:51.829975 11109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:51.830364 11107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:46:51.830704 11101 server_base.cc:1034] running on GCE node
I20240502 20:46:51.830996 11101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:46:51.831436 11101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:46:51.832573 11101 hybrid_clock.cc:648] HybridClock initialized: now 1714682811832557 us; error 29 us; skew 500 ppm
I20240502 20:46:51.834200 11101 webserver.cc:460] Webserver started at http://127.22.45.70:35361/ using document root <none> and password file <none>
I20240502 20:46:51.834697 11101 fs_manager.cc:359] Metadata directory not provided
I20240502 20:46:51.834760 11101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:46:51.837697 11101 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:51.839366 11115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:51.840303 11101 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:51.840391 11101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "e17061864e514564ba48c05dd80f7254"
format_stamp: "Formatted at 2024-05-02 20:46:42 on dist-test-slave-g7g5"
I20240502 20:46:51.840734 11101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:46:51.859630 11101 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:46:51.860108 11101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:46:51.860242 11101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:46:51.860687 11101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:46:51.861474 11101 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:46:51.861527 11101 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:51.861575 11101 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:46:51.861603 11101 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:51.870826 11101 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.70:37255
I20240502 20:46:51.870862 11231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.70:37255 every 8 connection(s)
I20240502 20:46:51.871716 11101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 20:46:51.873354 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 11101
I20240502 20:46:51.873780 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.71:44533
--local_ip_for_outbound_sockets=127.22.45.71
--tserver_master_addrs=127.22.45.126:41377
--webserver_port=33945
--webserver_interface=127.22.45.71
--builtin_ntp_servers=127.22.45.84:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:46:51.885145 11232 heartbeater.cc:347] Connected to a master server at 127.22.45.126:41377
I20240502 20:46:51.885382 11232 heartbeater.cc:464] Registering TS with master...
I20240502 20:46:51.885848 11232 heartbeater.cc:511] Master 127.22.45.126:41377 requested a full tablet report, sending...
I20240502 20:46:51.887003 9144 ts_manager.cc:194] Re-registered known tserver with Master: e17061864e514564ba48c05dd80f7254 (127.22.45.70:37255)
I20240502 20:46:51.887526 9144 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.70:43169
W20240502 20:46:51.977741 11235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:51.977906 11235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:46:51.977947 11235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:51.981077 11235 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:46:51.981132 11235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:46:51.981194 11235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.71
I20240502 20:46:51.984707 11235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.71:44533
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.22.45.71
--webserver_port=33945
--tserver_master_addrs=127.22.45.126:41377
--never_fsync=true
--heap_profile_path=/tmp/kudu.11235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.71
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:46:51.985368 11235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:46:51.986301 11235 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:46:51.993573 11241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:51.993956 11240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:51.994290 11243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:46:51.994685 11235 server_base.cc:1034] running on GCE node
I20240502 20:46:51.995085 11235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:46:51.995703 11235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:46:51.996852 11235 hybrid_clock.cc:648] HybridClock initialized: now 1714682811996830 us; error 33 us; skew 500 ppm
I20240502 20:46:51.998750 11235 webserver.cc:460] Webserver started at http://127.22.45.71:33945/ using document root <none> and password file <none>
I20240502 20:46:51.999222 11235 fs_manager.cc:359] Metadata directory not provided
I20240502 20:46:51.999296 11235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:46:52.002339 11235 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 20:46:52.004004 11250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:52.004894 11235 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 20:46:52.004995 11235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "d22b71ae7ed94ce295da0ca9955444d8"
format_stamp: "Formatted at 2024-05-02 20:46:42 on dist-test-slave-g7g5"
I20240502 20:46:52.005332 11235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:46:52.022279 11235 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:46:52.022768 11235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:46:52.022907 11235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:46:52.023344 11235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:46:52.024113 11235 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:46:52.024158 11235 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:52.024205 11235 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:46:52.024232 11235 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:52.032645 11235 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.71:44533
I20240502 20:46:52.032685 11366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.71:44533 every 8 connection(s)
I20240502 20:46:52.033529 11235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 20:46:52.041205 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 11235
I20240502 20:46:52.041452 11368 heartbeater.cc:347] Connected to a master server at 127.22.45.126:41377
I20240502 20:46:52.041586 22709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
/tmp/dist-test-taskzMQzMn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.45.72:43571
--local_ip_for_outbound_sockets=127.22.45.72
--tserver_master_addrs=127.22.45.126:41377
--webserver_port=42565
--webserver_interface=127.22.45.72
--builtin_ntp_servers=127.22.45.84:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 20:46:52.041647 11368 heartbeater.cc:464] Registering TS with master...
I20240502 20:46:52.042054 11368 heartbeater.cc:511] Master 127.22.45.126:41377 requested a full tablet report, sending...
I20240502 20:46:52.042951 9144 ts_manager.cc:194] Re-registered known tserver with Master: d22b71ae7ed94ce295da0ca9955444d8 (127.22.45.71:44533)
I20240502 20:46:52.043538 9144 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.71:51791
W20240502 20:46:52.143985 11371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 20:46:52.144156 11371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 20:46:52.144210 11371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 20:46:52.147208 11371 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 20:46:52.147266 11371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 20:46:52.147348 11371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.45.72
I20240502 20:46:52.150831 11371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.45.84:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.45.72:43571
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.22.45.72
--webserver_port=42565
--tserver_master_addrs=127.22.45.126:41377
--never_fsync=true
--heap_profile_path=/tmp/kudu.11371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.45.72
--log_dir=/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision fdb42489d76098eeca371ac57ae02ea176438c64
build type DEBUG
built by None at 02 May 2024 20:15:58 UTC on efd1337a2aa3
build id 93
I20240502 20:46:52.151536 11371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 20:46:52.152532 11371 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 20:46:52.159986 11377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:52.160334 11380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 20:46:52.160703 11378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 20:46:52.161092 11371 server_base.cc:1034] running on GCE node
I20240502 20:46:52.161422 11371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 20:46:52.161976 11371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 20:46:52.163131 11371 hybrid_clock.cc:648] HybridClock initialized: now 1714682812163084 us; error 62 us; skew 500 ppm
I20240502 20:46:52.164705 11371 webserver.cc:460] Webserver started at http://127.22.45.72:42565/ using document root <none> and password file <none>
I20240502 20:46:52.165180 11371 fs_manager.cc:359] Metadata directory not provided
I20240502 20:46:52.165252 11371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 20:46:52.168347 11371 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 20:46:52.170095 11386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:52.170998 11371 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 20:46:52.171094 11371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "e1107bea1a174997a62341f2c771f268"
format_stamp: "Formatted at 2024-05-02 20:46:42 on dist-test-slave-g7g5"
I20240502 20:46:52.171425 11371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 20:46:52.186244 11371 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 20:46:52.186758 11371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 20:46:52.186895 11371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 20:46:52.187348 11371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 20:46:52.188105 11371 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 20:46:52.188151 11371 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:52.188197 11371 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 20:46:52.188223 11371 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 20:46:52.197229 11371 rpc_server.cc:307] RPC server started. Bound to: 127.22.45.72:43571
I20240502 20:46:52.197288 11503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.45.72:43571 every 8 connection(s)
I20240502 20:46:52.198297 11371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzMQzMn/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714682725224411-22709-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 20:46:52.205729 11504 heartbeater.cc:347] Connected to a master server at 127.22.45.126:41377
I20240502 20:46:52.205919 11504 heartbeater.cc:464] Registering TS with master...
I20240502 20:46:52.206277 11504 heartbeater.cc:511] Master 127.22.45.126:41377 requested a full tablet report, sending...
I20240502 20:46:52.207026 22709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzMQzMn/build/debug/bin/kudu as pid 11371
I20240502 20:46:52.207268 22709 test_util.cc:268] Using random seed: -1276342111
I20240502 20:46:52.207278 9144 ts_manager.cc:194] Re-registered known tserver with Master: e1107bea1a174997a62341f2c771f268 (127.22.45.72:43571)
I20240502 20:46:52.207876 9144 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.45.72:39459
I20240502 20:46:52.217129 22709 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_0 already exists
I20240502 20:46:52.217332 22709 test_util.cc:268] Using random seed: -1276332046
I20240502 20:46:52.224923 9556 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:52.225082 9556 raft_consensus.cc:465] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240502 20:46:52.225713 22709 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_1 already exists
I20240502 20:46:52.225728 9556 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:52.225841 22709 test_util.cc:268] Using random seed: -1276323536
I20240502 20:46:52.225857 9556 raft_consensus.cc:482] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.225914 9556 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.226943 9556 raft_consensus.cc:504] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.227303 9556 leader_election.cc:288] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:52.227731 9288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.227795 9689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.227862 9288 raft_consensus.cc:3032] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.227913 9288 raft_consensus.cc:729] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.227909 9689 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.228066 9288 consensus_queue.cc:260] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.228351 9288 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.228883 9689 raft_consensus.cc:2454] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.229293 9288 raft_consensus.cc:2454] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.229372 9492 leader_election.cc:302] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:52.229534 10554 raft_consensus.cc:2781] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.229648 10554 raft_consensus.cc:686] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.229760 10554 consensus_queue.cc:237] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.229944 9556 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:52.230036 9556 raft_consensus.cc:482] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.230080 9556 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.230916 9556 raft_consensus.cc:504] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.231653 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.231778 9424 raft_consensus.cc:3032] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.231817 9424 raft_consensus.cc:729] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.231935 9288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.231976 9424 consensus_queue.cc:260] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.232048 9288 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.232236 9424 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.232903 9288 raft_consensus.cc:2454] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.233058 9556 leader_election.cc:288] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:52.233175 9424 raft_consensus.cc:2454] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.233321 9490 leader_election.cc:302] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:52.233474 11532 raft_consensus.cc:2781] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.233547 11532 raft_consensus.cc:686] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.233662 11532 consensus_queue.cc:237] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.234014 9556 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:52.234222 9556 raft_consensus.cc:482] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.234297 9556 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.235030 9556 raft_consensus.cc:504] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.235569 9556 leader_election.cc:288] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:52.235630 9689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.235723 9689 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.235766 9144 catalog_manager.cc:5484] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 1 to 2, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 2 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:52.236279 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.236382 9423 raft_consensus.cc:3032] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.236419 9423 raft_consensus.cc:729] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.236533 9423 consensus_queue.cc:260] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.236613 9689 raft_consensus.cc:2454] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.236682 9423 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.236740 22709 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_2 already exists
I20240502 20:46:52.236866 22709 test_util.cc:268] Using random seed: -1276312512
I20240502 20:46:52.236860 9492 leader_election.cc:302] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:52.237021 9556 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:52.237099 9556 raft_consensus.cc:482] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.237133 9556 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.237589 9423 raft_consensus.cc:2454] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.237705 9556 raft_consensus.cc:504] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.238178 9556 leader_election.cc:288] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:52.238225 9288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.238323 9288 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.238399 11532 raft_consensus.cc:2781] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.238551 11532 raft_consensus.cc:686] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.238675 11532 consensus_queue.cc:237] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.239187 9288 raft_consensus.cc:2454] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.239410 9689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.239514 9689 raft_consensus.cc:3032] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.239562 9689 raft_consensus.cc:729] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:52.239698 9689 consensus_queue.cc:260] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.239929 9689 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.240705 9689 raft_consensus.cc:2454] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.241256 9492 leader_election.cc:302] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:52.241699 11532 raft_consensus.cc:2781] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.241781 11532 raft_consensus.cc:686] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.241904 11532 consensus_queue.cc:237] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.242341 9557 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:52.242442 9557 raft_consensus.cc:465] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240502 20:46:52.243067 9557 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:52.243165 9557 raft_consensus.cc:482] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.243207 9557 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.244030 9557 raft_consensus.cc:504] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.244405 9557 leader_election.cc:288] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:52.245390 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.245491 9423 raft_consensus.cc:3032] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.245527 9423 raft_consensus.cc:729] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.245635 9423 consensus_queue.cc:260] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.245779 9423 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.246639 9423 raft_consensus.cc:2454] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.246843 9288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.246940 9288 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.247701 9288 raft_consensus.cc:2454] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.247973 9490 leader_election.cc:302] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:52.248430 11532 raft_consensus.cc:2781] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.248520 11532 raft_consensus.cc:686] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.248631 11532 consensus_queue.cc:237] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.248832 9557 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3315525f1849462fbf49a285f424ea51"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:52.248921 9557 raft_consensus.cc:482] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.248968 9557 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.249650 9557 raft_consensus.cc:504] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.249903 9557 leader_election.cc:288] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:52.249919 9144 catalog_manager.cc:5484] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 1 to 2, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 2 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:52.250133 9144 catalog_manager.cc:5484] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 1 to 2, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 2 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:52.250319 9144 catalog_manager.cc:5484] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 1 to 2, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 2 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:52.250748 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3315525f1849462fbf49a285f424ea51" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.250849 9424 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.251034 9288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3315525f1849462fbf49a285f424ea51" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.251128 9288 raft_consensus.cc:3032] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.251163 9288 raft_consensus.cc:729] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.251303 9288 consensus_queue.cc:260] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.251459 9288 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Advancing to term 2
W20240502 20:46:52.251616 11558 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59728: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.128s ago.
I20240502 20:46:52.251649 9424 raft_consensus.cc:2454] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.252231 9288 raft_consensus.cc:2454] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.252511 9490 leader_election.cc:302] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:52.252781 11532 raft_consensus.cc:2781] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.252857 11532 raft_consensus.cc:686] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.252965 11532 consensus_queue.cc:237] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.253124 9557 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:52.253492 9557 raft_consensus.cc:482] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.253563 9557 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.254240 9557 raft_consensus.cc:504] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.254592 9557 leader_election.cc:288] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:52.254897 9288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.254994 9288 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.255790 9288 raft_consensus.cc:2454] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.256012 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.256121 9423 raft_consensus.cc:3032] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.256167 9423 raft_consensus.cc:729] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.256188 9556 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:52.256281 9556 raft_consensus.cc:465] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240502 20:46:52.256296 9490 leader_election.cc:302] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:52.256289 9423 consensus_queue.cc:260] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.256441 9423 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.256515 11532 raft_consensus.cc:2781] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.256600 11532 raft_consensus.cc:686] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.256705 11532 consensus_queue.cc:237] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.257138 9423 raft_consensus.cc:2454] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.258249 9556 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "edebc185b24441da9504c78394378f4d"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:52.258353 9556 raft_consensus.cc:482] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.258401 9556 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.259163 9556 raft_consensus.cc:504] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.259510 9556 leader_election.cc:288] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:52.259986 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edebc185b24441da9504c78394378f4d" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.260087 9688 raft_consensus.cc:3032] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.260114 9688 raft_consensus.cc:729] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:52.260219 9688 consensus_queue.cc:260] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.260385 9688 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.261211 9688 raft_consensus.cc:2454] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.261529 9492 leader_election.cc:302] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:52.261679 11532 raft_consensus.cc:2781] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.261755 11532 raft_consensus.cc:686] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.261870 11532 consensus_queue.cc:237] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.263070 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edebc185b24441da9504c78394378f4d" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.263165 9423 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Advancing to term 2
W20240502 20:46:52.263648 9382 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55522: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:52.263911 9423 raft_consensus.cc:2454] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.264402 9556 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:52.264488 9556 raft_consensus.cc:465] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240502 20:46:52.265013 9556 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:52.265108 9556 raft_consensus.cc:465] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240502 20:46:52.265597 9556 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:52.265699 9556 raft_consensus.cc:482] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.265759 9556 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.266628 9556 raft_consensus.cc:504] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.267313 9556 leader_election.cc:288] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:52.268004 9556 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "20698b2531674eefb9df3125a86653ba"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:52.268065 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.268101 9556 raft_consensus.cc:482] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.268153 9556 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.268160 9289 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.268896 9556 raft_consensus.cc:504] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.269008 9289 raft_consensus.cc:2454] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.269381 9556 leader_election.cc:288] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:52.269935 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20698b2531674eefb9df3125a86653ba" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.270025 9289 raft_consensus.cc:3032] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.270067 9289 raft_consensus.cc:729] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.270195 9289 consensus_queue.cc:260] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.270345 9289 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.271082 9289 raft_consensus.cc:2454] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.271459 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20698b2531674eefb9df3125a86653ba" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.271553 9423 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.272321 9423 raft_consensus.cc:2454] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.272389 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.272475 9688 raft_consensus.cc:3032] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.272517 9688 raft_consensus.cc:729] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:52.272644 9688 consensus_queue.cc:260] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.272719 22709 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_3 already exists
I20240502 20:46:52.272784 9688 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.272876 22709 test_util.cc:268] Using random seed: -1276276502
I20240502 20:46:52.273617 9688 raft_consensus.cc:2454] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.273864 9492 leader_election.cc:302] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:52.274022 11532 raft_consensus.cc:2781] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.274071 9491 leader_election.cc:302] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:52.274231 10554 raft_consensus.cc:2781] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.274392 10554 raft_consensus.cc:686] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.274538 10554 consensus_queue.cc:237] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.274091 11532 raft_consensus.cc:686] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.275197 11532 consensus_queue.cc:237] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.276266 9556 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:52.276355 9556 raft_consensus.cc:482] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.276409 9556 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.276497 9289 raft_consensus.cc:1261] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:52.276561 9423 raft_consensus.cc:1261] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:52.276906 11532 consensus_queue.cc:1035] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.277108 9556 raft_consensus.cc:504] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.277168 11532 consensus_queue.cc:1035] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.277439 9556 leader_election.cc:288] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:52.279263 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.279357 9424 raft_consensus.cc:3032] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.279392 9424 raft_consensus.cc:729] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.279515 9424 consensus_queue.cc:260] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.279675 9424 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.279871 9136 catalog_manager.cc:5484] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 1 to 2, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 2 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:52.280467 9424 raft_consensus.cc:2454] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.280926 9491 leader_election.cc:302] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:52.281119 10554 raft_consensus.cc:2781] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.281191 10554 raft_consensus.cc:686] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.281292 10554 consensus_queue.cc:237] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.282086 9556 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:52.282192 9556 raft_consensus.cc:482] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.282243 9556 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.282984 9556 raft_consensus.cc:504] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.283293 9556 leader_election.cc:288] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:52.285760 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.285861 9688 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.286296 22709 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_4 already exists
I20240502 20:46:52.286360 9288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.286423 22709 test_util.cc:268] Using random seed: -1276262955
I20240502 20:46:52.286450 9288 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.286707 9688 raft_consensus.cc:2454] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.287271 9288 raft_consensus.cc:2454] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.287557 9490 leader_election.cc:302] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:52.287720 11532 raft_consensus.cc:2781] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.287797 11532 raft_consensus.cc:686] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.287912 11532 consensus_queue.cc:237] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.288492 9556 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:52.288594 9556 raft_consensus.cc:482] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.288643 9556 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.290951 9142 catalog_manager.cc:5484] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 1 to 2, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 2 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:52.291175 9142 catalog_manager.cc:5484] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 1 to 2, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 2 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:52.291344 9142 catalog_manager.cc:5484] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 1 to 2, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 2 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:52.291517 9142 catalog_manager.cc:5484] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 1 to 2, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 2 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:52.291694 9142 catalog_manager.cc:5484] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 1 to 2, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 2 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:52.291774 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.291858 9424 raft_consensus.cc:3032] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.291893 9424 raft_consensus.cc:729] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.291872 9142 catalog_manager.cc:5484] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 1 to 2, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 2 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:52.292006 9424 consensus_queue.cc:260] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.292039 9142 catalog_manager.cc:5484] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 1 to 2, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 2 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:52.292176 9424 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.292997 9424 raft_consensus.cc:2454] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.293601 9556 raft_consensus.cc:504] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.293937 9556 leader_election.cc:288] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:52.295034 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.295148 9687 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.295213 9556 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:52.295325 9556 raft_consensus.cc:482] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.295379 9556 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.296005 9687 raft_consensus.cc:2454] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.296124 9556 raft_consensus.cc:504] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.296483 9556 leader_election.cc:288] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:52.297495 9288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.297591 9288 raft_consensus.cc:3032] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.297629 9288 raft_consensus.cc:729] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.297745 9288 consensus_queue.cc:260] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.297896 9288 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.298750 9288 raft_consensus.cc:2454] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.298731 9492 leader_election.cc:302] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:52.298913 10554 raft_consensus.cc:2781] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.298995 10554 raft_consensus.cc:686] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.299115 10554 consensus_queue.cc:237] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.300956 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.301054 9424 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.301149 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.301242 9688 raft_consensus.cc:3032] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.301277 9688 raft_consensus.cc:729] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:52.301391 9688 consensus_queue.cc:260] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.301539 9688 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.301826 9424 raft_consensus.cc:2454] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.301993 9556 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:52.302088 9556 raft_consensus.cc:482] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.302150 9556 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.302371 9688 raft_consensus.cc:2454] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.302665 9491 leader_election.cc:302] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:52.302901 9556 raft_consensus.cc:504] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.303136 9142 catalog_manager.cc:5484] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 1 to 2, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 2 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:52.303223 9556 leader_election.cc:288] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:52.303738 10554 raft_consensus.cc:2781] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.303812 10554 raft_consensus.cc:686] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.303853 9556 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:52.303922 10554 consensus_queue.cc:237] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.304317 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.304428 9686 raft_consensus.cc:3032] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.304478 9686 raft_consensus.cc:729] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:52.304621 9686 consensus_queue.cc:260] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.304775 9686 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.304920 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.305017 9424 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.305584 9686 raft_consensus.cc:2454] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.305745 9424 raft_consensus.cc:2454] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.306020 9491 leader_election.cc:302] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:52.306174 11577 raft_consensus.cc:2781] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.306253 11577 raft_consensus.cc:686] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.306376 11577 consensus_queue.cc:237] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.303929 9556 raft_consensus.cc:482] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.307412 9556 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.308161 9556 raft_consensus.cc:504] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.308458 9556 leader_election.cc:288] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:52.309474 9556 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:52.309581 9556 raft_consensus.cc:465] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240502 20:46:52.312011 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.312134 9286 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.312952 9286 raft_consensus.cc:2454] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.313740 9490 leader_election.cc:302] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:52.313901 11577 raft_consensus.cc:2781] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.313972 11577 raft_consensus.cc:686] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.314080 11577 consensus_queue.cc:237] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
W20240502 20:46:52.314749 9382 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55530: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:52.315256 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.315358 9424 raft_consensus.cc:3032] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.315395 9424 raft_consensus.cc:729] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.315510 9424 consensus_queue.cc:260] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.315654 9424 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.316430 9424 raft_consensus.cc:2454] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 2.
I20240502 20:46:52.319061 9142 catalog_manager.cc:5484] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 1 to 2, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 2 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:52.319273 9142 catalog_manager.cc:5484] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 1 to 2, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 2 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:52.319461 9142 catalog_manager.cc:5484] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 1 to 2, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 2 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:52.319509 9556 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:52.319604 9556 raft_consensus.cc:465] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240502 20:46:52.325289 9421 raft_consensus.cc:1261] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:52.325834 9288 raft_consensus.cc:1261] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:52.326195 11532 consensus_queue.cc:1035] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.326809 11532 consensus_queue.cc:1035] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.332397 11539 mvcc.cc:205] Tried to move back new op lower bound from 7023340799280410624 to 7023340799132397568. Current Snapshot: MvccSnapshot[applied={T|T < 7023340799280410624}]
I20240502 20:46:52.337436 9423 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:52.337560 9423 raft_consensus.cc:482] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.337618 9423 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.338552 9423 raft_consensus.cc:504] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.338896 9423 leader_election.cc:288] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 3 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:52.339749 9288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.339869 9288 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.339892 9556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.339908 9423 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:52.339989 9556 raft_consensus.cc:3032] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:52.339988 9423 raft_consensus.cc:465] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240502 20:46:52.340023 9556 raft_consensus.cc:729] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.340159 9556 consensus_queue.cc:260] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.340381 9556 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.340767 9288 raft_consensus.cc:2454] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 3.
I20240502 20:46:52.341266 9556 raft_consensus.cc:2396] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 20:46:52.341732 9355 leader_election.cc:302] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e
I20240502 20:46:52.341950 11585 raft_consensus.cc:2781] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Leader election won for term 3
I20240502 20:46:52.342025 11585 raft_consensus.cc:686] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.342137 11585 consensus_queue.cc:237] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.343461 9422 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:52.343564 9422 raft_consensus.cc:482] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.343616 9422 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.344497 9422 raft_consensus.cc:504] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.344806 9422 leader_election.cc:288] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 3 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:52.345024 9142 catalog_manager.cc:5484] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 2 to 3, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 3 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:52.345445 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.345541 9688 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.346390 9688 raft_consensus.cc:2454] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 3.
I20240502 20:46:52.346709 9357 leader_election.cc:302] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:52.346966 10462 raft_consensus.cc:2781] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Leader election won for term 3
I20240502 20:46:52.347043 10462 raft_consensus.cc:686] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.347147 10462 consensus_queue.cc:237] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
W20240502 20:46:52.347661 9647 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40084: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:52.347796 9423 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "edebc185b24441da9504c78394378f4d"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:52.347879 9423 raft_consensus.cc:482] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.347924 9423 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.348647 9423 raft_consensus.cc:504] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.348953 9423 leader_election.cc:288] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 3 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:52.349330 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edebc185b24441da9504c78394378f4d" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 3 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.349431 9687 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.349671 9423 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:52.349754 9423 raft_consensus.cc:482] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.349807 9423 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.350266 9687 raft_consensus.cc:2454] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 3.
I20240502 20:46:52.350517 9423 raft_consensus.cc:504] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.350558 9557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edebc185b24441da9504c78394378f4d" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 3 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.350642 9557 raft_consensus.cc:3032] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:52.350674 9557 raft_consensus.cc:729] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.350775 9557 consensus_queue.cc:260] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.6, Last appended by leader: 6, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.350896 9557 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.351065 9288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.351146 9288 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.351759 9557 raft_consensus.cc:2396] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 3 because replica has last-logged OpId of term: 2 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 5.
I20240502 20:46:52.351907 9288 raft_consensus.cc:2454] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 3.
I20240502 20:46:52.352186 9355 leader_election.cc:302] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:52.352335 11585 raft_consensus.cc:2781] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Leader election won for term 3
I20240502 20:46:52.352403 11585 raft_consensus.cc:686] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.352509 11585 consensus_queue.cc:237] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.352777 9357 leader_election.cc:302] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:52.353003 11585 raft_consensus.cc:2781] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Leader election won for term 3
I20240502 20:46:52.353068 11585 raft_consensus.cc:686] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.353168 11585 consensus_queue.cc:237] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.353278 9556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.353370 9556 raft_consensus.cc:3032] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:52.353417 9556 raft_consensus.cc:729] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.353574 9556 consensus_queue.cc:260] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.353706 9556 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.354548 9556 raft_consensus.cc:2396] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
W20240502 20:46:52.354650 9647 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:52.356010 9688 raft_consensus.cc:1261] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 3 index: 7. (index mismatch)
I20240502 20:46:52.356539 10462 consensus_queue.cc:1035] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 20:46:52.357133 9556 raft_consensus.cc:1261] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 3 index: 7. (index mismatch)
I20240502 20:46:52.357916 9423 leader_election.cc:288] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 3 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:52.358017 9142 catalog_manager.cc:5484] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 2 to 3, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 3 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
W20240502 20:46:52.359206 9382 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:52.361721 10462 consensus_queue.cc:1035] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 20:46:52.361825 10462 consensus_queue.cc:1225] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Peer 190c1d3968d949bbbaa27a51a2b91d8e log is divergent from this leader: its last log entry 2.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20240502 20:46:52.362838 9142 catalog_manager.cc:5484] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 2 to 3, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 3 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:52.363023 9142 catalog_manager.cc:5484] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 2 to 3, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 3 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:52.363867 9424 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3315525f1849462fbf49a285f424ea51"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:52.363958 9424 raft_consensus.cc:482] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.364006 9424 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.364701 9424 raft_consensus.cc:504] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.365012 9424 leader_election.cc:288] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 3 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:52.366298 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3315525f1849462fbf49a285f424ea51" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.366315 9424 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:52.366398 9289 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.366395 9424 raft_consensus.cc:482] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.366443 9424 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.367201 9424 raft_consensus.cc:504] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.367262 9289 raft_consensus.cc:2454] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 3.
I20240502 20:46:52.367514 9424 leader_election.cc:288] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 3 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:52.368763 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.368863 9687 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.369331 9424 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:52.369447 9424 raft_consensus.cc:482] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.369505 9424 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.369690 9687 raft_consensus.cc:2454] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 3.
I20240502 20:46:52.370234 9424 raft_consensus.cc:504] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.370347 9355 leader_election.cc:302] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:52.370581 9424 leader_election.cc:288] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 3 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:52.370954 9556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.371053 9357 leader_election.cc:302] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:52.371071 9556 raft_consensus.cc:3032] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:52.371124 9556 raft_consensus.cc:729] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.371215 11585 raft_consensus.cc:2781] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Leader election won for term 3
I20240502 20:46:52.371235 9556 consensus_queue.cc:260] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.371310 11585 raft_consensus.cc:686] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.371366 9556 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.371425 11585 consensus_queue.cc:237] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.371904 11585 raft_consensus.cc:2781] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Leader election won for term 3
I20240502 20:46:52.371982 11585 raft_consensus.cc:686] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.372116 11585 consensus_queue.cc:237] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.372421 9556 raft_consensus.cc:2396] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 20:46:52.372906 9556 pending_rounds.cc:76] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e: Aborting all ops after (but not including) 5
I20240502 20:46:52.372959 9556 pending_rounds.cc:97] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e: Aborting uncommitted NO_OP operation due to leader change: 2.6
I20240502 20:46:52.372969 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.373014 9556 raft_consensus.cc:2866] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 20:46:52.373064 9289 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.373245 9557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3315525f1849462fbf49a285f424ea51" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.373329 9557 raft_consensus.cc:3032] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:52.373361 9557 raft_consensus.cc:729] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.373467 9557 consensus_queue.cc:260] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.373603 9557 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.373886 9289 raft_consensus.cc:2454] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 3.
I20240502 20:46:52.374089 9136 catalog_manager.cc:5484] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 2 to 3, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 3 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:52.374302 9136 catalog_manager.cc:5484] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 2 to 3, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 3 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:52.374439 9557 raft_consensus.cc:2454] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 3.
I20240502 20:46:52.375367 9557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.375453 9557 raft_consensus.cc:3032] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:52.375484 9557 raft_consensus.cc:729] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.375581 9557 consensus_queue.cc:260] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.375725 9557 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.376188 9355 leader_election.cc:302] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:52.376368 11628 raft_consensus.cc:2781] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Leader election won for term 3
I20240502 20:46:52.376446 11628 raft_consensus.cc:686] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.376495 9557 raft_consensus.cc:2396] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 20:46:52.376549 11628 consensus_queue.cc:237] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.377204 9423 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:52.377300 9423 raft_consensus.cc:482] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.377346 9423 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.377625 9557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.377722 9557 raft_consensus.cc:3032] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:52.377758 9557 raft_consensus.cc:729] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.377868 9557 consensus_queue.cc:260] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.378018 9557 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.378108 9423 raft_consensus.cc:504] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.378407 9423 leader_election.cc:288] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 2 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:52.378885 9557 raft_consensus.cc:2454] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 3.
I20240502 20:46:52.379406 9557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.379506 9557 raft_consensus.cc:3032] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.379551 9557 raft_consensus.cc:729] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.379658 9557 consensus_queue.cc:260] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.379685 9423 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fe22224142984d69a2def286584b9aaa"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:52.379789 9557 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.379779 9423 raft_consensus.cc:482] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.379841 9423 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.380198 9136 catalog_manager.cc:5484] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 2 to 3, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 3 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:52.380492 9557 raft_consensus.cc:2454] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 2.
I20240502 20:46:52.380566 9423 raft_consensus.cc:504] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.380885 9423 leader_election.cc:288] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 2 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:52.381006 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.381517 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe22224142984d69a2def286584b9aaa" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.381618 9286 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.381947 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe22224142984d69a2def286584b9aaa" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.382032 9688 raft_consensus.cc:3032] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.382063 9688 raft_consensus.cc:729] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:52.382170 9688 consensus_queue.cc:260] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.382318 9688 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.382491 9286 raft_consensus.cc:2454] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 2.
I20240502 20:46:52.383682 9355 leader_election.cc:302] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
W20240502 20:46:52.383944 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51610: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:52.384302 9357 leader_election.cc:302] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:52.384495 10462 raft_consensus.cc:2781] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.384567 10462 raft_consensus.cc:686] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 2 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.384672 10462 consensus_queue.cc:237] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.385147 10462 raft_consensus.cc:2781] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.385221 10462 raft_consensus.cc:686] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.385321 10462 consensus_queue.cc:237] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.387069 9688 raft_consensus.cc:2454] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 2.
I20240502 20:46:52.387133 9142 catalog_manager.cc:5484] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 1 to 2, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 2 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:52.387319 9142 catalog_manager.cc:5484] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 1 to 2, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 2 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
W20240502 20:46:52.389379 9647 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40084: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:52.390863 9423 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:52.390965 9423 raft_consensus.cc:482] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.391014 9423 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.391760 22709 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_5 already exists
I20240502 20:46:52.391772 9423 raft_consensus.cc:504] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.391940 22709 test_util.cc:268] Using random seed: -1276157439
I20240502 20:46:52.392074 9423 leader_election.cc:288] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 2 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:52.392236 9689 raft_consensus.cc:1261] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:52.392685 11579 consensus_queue.cc:1035] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.394533 9289 raft_consensus.cc:1261] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 7. (index mismatch)
I20240502 20:46:52.395157 10462 consensus_queue.cc:1035] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20240502 20:46:52.397570 9248 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59742: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:52.398591 9557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.398738 9557 raft_consensus.cc:3032] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.398823 9557 raft_consensus.cc:729] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.399019 9557 consensus_queue.cc:260] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.399271 9557 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.400133 9557 raft_consensus.cc:2454] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 2.
I20240502 20:46:52.400452 9556 raft_consensus.cc:1261] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 7. (index mismatch)
I20240502 20:46:52.401136 9423 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:52.401234 9423 raft_consensus.cc:482] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.401295 9423 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.402051 9423 raft_consensus.cc:504] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.402364 9423 leader_election.cc:288] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 2 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:52.402668 11607 mvcc.cc:205] Tried to move back new op lower bound from 7023340799481450496 to 7023340799198134272. Current Snapshot: MvccSnapshot[applied={T|T < 7023340799481450496 or (T in {7023340799481450496})}]
I20240502 20:46:52.402822 9357 leader_election.cc:302] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:52.402931 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.402977 11585 raft_consensus.cc:2781] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.403038 11585 raft_consensus.cc:686] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.403038 9287 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.403147 11585 consensus_queue.cc:237] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.403841 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.403872 9287 raft_consensus.cc:2454] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 2.
I20240502 20:46:52.403940 9688 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.403939 9423 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:52.404029 9423 raft_consensus.cc:482] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.404094 9423 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.404785 9688 raft_consensus.cc:2454] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 2.
I20240502 20:46:52.404784 9423 raft_consensus.cc:504] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.405095 9689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.405112 9423 leader_election.cc:288] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 3 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:52.405205 9689 raft_consensus.cc:3032] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.405244 9689 raft_consensus.cc:729] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:52.405730 9287 raft_consensus.cc:1261] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240502 20:46:52.405872 9423 raft_consensus.cc:1261] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:52.405939 9136 catalog_manager.cc:5484] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 1 to 2, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 2 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:52.406234 10554 consensus_queue.cc:1035] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.406819 9355 leader_election.cc:302] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:52.406924 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.407009 9286 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.407204 11585 raft_consensus.cc:2781] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.407363 11585 raft_consensus.cc:686] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.407486 11585 consensus_queue.cc:237] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.407891 9286 raft_consensus.cc:2454] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 3.
I20240502 20:46:52.407999 11585 consensus_queue.cc:1035] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 20:46:52.408239 9355 leader_election.cc:302] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:52.408504 10462 raft_consensus.cc:2781] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Leader election won for term 3
I20240502 20:46:52.408571 10462 raft_consensus.cc:686] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.408668 10462 consensus_queue.cc:237] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.408777 9689 consensus_queue.cc:260] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.408939 9689 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.411314 9136 catalog_manager.cc:5484] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 1 to 2, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 2 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:52.411527 9136 catalog_manager.cc:5484] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 2 to 3, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 3 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:52.411801 11585 consensus_queue.cc:1035] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 20:46:52.412046 9422 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:52.412147 9422 raft_consensus.cc:482] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.412196 9422 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.412366 9689 raft_consensus.cc:2454] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 2.
I20240502 20:46:52.412725 9687 raft_consensus.cc:1261] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240502 20:46:52.413022 9422 raft_consensus.cc:504] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.413187 10462 consensus_queue.cc:1035] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 20:46:52.413334 9422 leader_election.cc:288] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 2 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:52.414556 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.414656 9686 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.415503 9686 raft_consensus.cc:2454] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 2.
W20240502 20:46:52.418602 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51626: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:52.419049 11558 mvcc.cc:205] Tried to move back new op lower bound from 7023340799607349248 to 7023340799528951808. Current Snapshot: MvccSnapshot[applied={T|T < 7023340799607349248 or (T in {7023340799607349248})}]
W20240502 20:46:52.419973 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51628: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:52.420115 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.420208 9555 raft_consensus.cc:3032] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:52.420251 9555 raft_consensus.cc:729] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.420374 9555 consensus_queue.cc:260] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.420504 9555 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.420619 9357 leader_election.cc:302] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:52.420801 11585 raft_consensus.cc:2781] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.420877 11585 raft_consensus.cc:686] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 2 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.420984 11585 consensus_queue.cc:237] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.421391 9555 raft_consensus.cc:2396] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 20:46:52.422981 9422 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:52.422927 9136 catalog_manager.cc:5484] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 1 to 2, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 2 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:52.423079 9422 raft_consensus.cc:482] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.423125 9422 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.423903 9422 raft_consensus.cc:504] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.423957 9557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.424094 9557 raft_consensus.cc:3032] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.424170 9557 raft_consensus.cc:729] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.424208 9422 leader_election.cc:288] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 2 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:52.424337 9557 consensus_queue.cc:260] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.424499 9557 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.425295 9557 raft_consensus.cc:2454] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 2.
W20240502 20:46:52.426085 9248 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59742: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:52.428290 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.428392 9286 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.430652 9289 raft_consensus.cc:1261] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 7. (index mismatch)
I20240502 20:46:52.431002 11639 consensus_queue.cc:1035] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 20:46:52.431807 9286 raft_consensus.cc:2454] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 2.
I20240502 20:46:52.432123 9355 leader_election.cc:302] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:52.432344 11638 raft_consensus.cc:2781] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.437301 11638 raft_consensus.cc:686] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 2 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.437462 11638 consensus_queue.cc:237] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
W20240502 20:46:52.437530 9647 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:52.439419 9142 catalog_manager.cc:5484] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 1 to 2, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 2 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:52.446758 22709 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_6 already exists
I20240502 20:46:52.446907 22709 test_util.cc:268] Using random seed: -1276102471
I20240502 20:46:52.449983 9685 raft_consensus.cc:1261] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:52.450152 9288 raft_consensus.cc:1261] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:52.450345 10554 consensus_queue.cc:1035] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.450773 10554 consensus_queue.cc:1035] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.452406 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.452512 9686 raft_consensus.cc:3032] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.452550 9686 raft_consensus.cc:729] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:52.452667 9686 consensus_queue.cc:260] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.452827 9686 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.453769 9686 raft_consensus.cc:2454] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 2.
W20240502 20:46:52.455128 9248 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59752: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:52.459751 9557 raft_consensus.cc:1261] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240502 20:46:52.459744 9686 raft_consensus.cc:1261] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240502 20:46:52.460351 9557 raft_consensus.cc:1261] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 7. (index mismatch)
I20240502 20:46:52.460669 9423 raft_consensus.cc:1261] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:52.460815 9286 raft_consensus.cc:1261] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:52.461061 10554 consensus_queue.cc:1035] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.461165 11532 consensus_queue.cc:1035] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.463162 11639 consensus_queue.cc:1035] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 20:46:52.463908 11639 consensus_queue.cc:1035] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 20:46:52.466114 9286 raft_consensus.cc:1261] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240502 20:46:52.466410 11638 consensus_queue.cc:1035] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 20:46:52.469033 9686 raft_consensus.cc:1261] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240502 20:46:52.469604 22709 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_7 already exists
I20240502 20:46:52.469741 22709 test_util.cc:268] Using random seed: -1276079636
I20240502 20:46:52.471312 9422 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:52.471421 9422 raft_consensus.cc:482] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.471473 9422 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.472417 9422 raft_consensus.cc:504] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.473026 9288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.473148 9288 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.474066 9288 raft_consensus.cc:2454] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 2.
I20240502 20:46:52.474532 9355 leader_election.cc:302] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:52.475601 9422 leader_election.cc:288] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 2 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:52.475749 11628 raft_consensus.cc:2781] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.475914 11628 raft_consensus.cc:686] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 2 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.476035 11628 consensus_queue.cc:237] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.477144 11585 consensus_queue.cc:1035] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 20:46:52.478152 9136 catalog_manager.cc:5484] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 1 to 2, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 2 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:52.482687 11585 consensus_queue.cc:1035] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 20:46:52.484083 9557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.484194 9557 raft_consensus.cc:3032] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.484241 9557 raft_consensus.cc:729] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.484375 9557 consensus_queue.cc:260] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.484510 9557 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.485332 9557 raft_consensus.cc:2454] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 2.
I20240502 20:46:52.498677 9423 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:52.498791 9423 raft_consensus.cc:482] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.498847 9423 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.499751 9423 raft_consensus.cc:504] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.500080 9423 leader_election.cc:288] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 3 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:52.500967 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.501081 9285 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.501757 9557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.501868 9557 raft_consensus.cc:3032] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:52.501911 9557 raft_consensus.cc:729] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.501996 9285 raft_consensus.cc:2454] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 3.
I20240502 20:46:52.502025 9557 consensus_queue.cc:260] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.502146 9557 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.503055 9557 raft_consensus.cc:2396] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 20:46:52.510943 9423 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d92bfee8fabd46a28ffcb7e319879406"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:52.511054 9423 raft_consensus.cc:465] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240502 20:46:52.511233 22709 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_8 already exists
I20240502 20:46:52.511384 22709 test_util.cc:268] Using random seed: -1276037996
I20240502 20:46:52.513211 9423 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:52.513327 9423 raft_consensus.cc:482] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.513381 9423 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.514218 9423 raft_consensus.cc:504] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.514565 9423 leader_election.cc:288] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 2 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:52.514919 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.515034 9688 raft_consensus.cc:3032] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.515085 9688 raft_consensus.cc:729] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
W20240502 20:46:52.515233 9647 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40118: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:52.515210 9688 consensus_queue.cc:260] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.515372 9688 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.515990 9355 leader_election.cc:302] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e
I20240502 20:46:52.516505 9688 raft_consensus.cc:2454] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 2.
I20240502 20:46:52.516657 9284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.516754 9284 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.516928 10462 raft_consensus.cc:2781] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Leader election won for term 3
I20240502 20:46:52.517010 10462 raft_consensus.cc:686] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.517099 10462 consensus_queue.cc:237] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.517665 9284 raft_consensus.cc:2454] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 2.
I20240502 20:46:52.519302 9144 catalog_manager.cc:5484] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 2 to 3, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 3 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
W20240502 20:46:52.522266 9248 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59728: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:52.523350 9357 leader_election.cc:302] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:52.523785 11628 raft_consensus.cc:2781] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.524060 11628 raft_consensus.cc:686] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 2 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.524189 11628 consensus_queue.cc:237] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.525935 9144 catalog_manager.cc:5484] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 1 to 2, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 2 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:52.528427 9423 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "20698b2531674eefb9df3125a86653ba"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:52.528527 9423 raft_consensus.cc:482] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.528577 9423 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.530756 9689 raft_consensus.cc:1261] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:52.531201 10554 consensus_queue.cc:1035] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.536615 9423 raft_consensus.cc:504] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.536907 9423 leader_election.cc:288] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 3 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:52.537153 9284 raft_consensus.cc:1261] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:52.538035 22709 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_9 already exists
I20240502 20:46:52.538113 9423 raft_consensus.cc:1261] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:52.538225 9688 raft_consensus.cc:1261] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:52.538497 11578 consensus_queue.cc:1035] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.538610 10554 consensus_queue.cc:1035] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.541867 9557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20698b2531674eefb9df3125a86653ba" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.541977 9557 raft_consensus.cc:3032] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:52.542021 9557 raft_consensus.cc:729] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.542136 9557 consensus_queue.cc:260] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 7, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.542271 9557 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.543212 9557 raft_consensus.cc:2454] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 3.
I20240502 20:46:52.544292 9357 leader_election.cc:302] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:52.544600 11638 raft_consensus.cc:2781] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Leader election won for term 3
I20240502 20:46:52.544679 11638 raft_consensus.cc:686] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.544800 11638 consensus_queue.cc:237] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 7, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.545260 11629 mvcc.cc:205] Tried to move back new op lower bound from 7023340800153260032 to 7023340799208185856. Current Snapshot: MvccSnapshot[applied={T|T < 7023340800153260032}]
I20240502 20:46:52.545291 9423 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:52.545380 9423 raft_consensus.cc:482] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.545423 9423 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.546233 9423 raft_consensus.cc:504] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.546559 9423 leader_election.cc:288] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 3 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:52.547048 11532 consensus_queue.cc:1035] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.547809 9556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.547906 9556 raft_consensus.cc:3032] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:52.547943 9556 raft_consensus.cc:729] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.548060 9556 consensus_queue.cc:260] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 8, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.548197 9556 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.549073 9556 raft_consensus.cc:2454] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 3.
I20240502 20:46:52.549618 9423 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:52.549723 9423 raft_consensus.cc:482] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.549787 9423 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.550586 9423 raft_consensus.cc:504] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.550881 9423 leader_election.cc:288] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 3 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:52.551302 9136 catalog_manager.cc:5484] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 2 to 3, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 3 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:52.552867 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.552986 9687 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.553941 9687 raft_consensus.cc:2454] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 3.
I20240502 20:46:52.554075 9689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.554105 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20698b2531674eefb9df3125a86653ba" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.554184 9689 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.555107 9689 raft_consensus.cc:2454] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 3.
I20240502 20:46:52.558267 9689 raft_consensus.cc:1261] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:52.558353 9687 raft_consensus.cc:1226] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Rejecting Update request from peer 190c1d3968d949bbbaa27a51a2b91d8e for earlier term 2. Current term is 3. Ops: []
I20240502 20:46:52.559798 9357 leader_election.cc:302] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:52.559813 9285 raft_consensus.cc:1261] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:52.560448 10554 consensus_queue.cc:1035] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.560693 10554 consensus_queue.cc:1035] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.560948 9557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.561040 9557 raft_consensus.cc:3032] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:52.561077 9557 raft_consensus.cc:729] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.561192 9557 consensus_queue.cc:260] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.561542 11532 consensus_queue.cc:1046] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.561702 11638 raft_consensus.cc:2781] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Leader election won for term 3
I20240502 20:46:52.561782 11638 raft_consensus.cc:686] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.561909 11638 consensus_queue.cc:237] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 8, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.561968 9557 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.562844 9557 raft_consensus.cc:2454] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 3.
I20240502 20:46:52.563779 9136 catalog_manager.cc:5484] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 2 to 3, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 3 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:52.564268 9357 leader_election.cc:302] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:52.564427 10462 raft_consensus.cc:2781] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Leader election won for term 3
I20240502 20:46:52.564502 10462 raft_consensus.cc:686] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.564605 10462 consensus_queue.cc:237] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.567425 9689 raft_consensus.cc:1261] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:52.567516 9423 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:52.567612 9423 raft_consensus.cc:482] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.567660 9423 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.568485 9423 raft_consensus.cc:504] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.569869 9144 catalog_manager.cc:5484] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 2 to 3, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 3 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:52.570232 9287 raft_consensus.cc:1261] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:52.570307 9423 leader_election.cc:288] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 2 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:52.571449 11628 consensus_queue.cc:1035] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 20:46:52.572592 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51656: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:52.574066 9689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.574219 9689 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.575138 11638 consensus_queue.cc:1035] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.575580 9283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.575604 9689 raft_consensus.cc:2454] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 2.
I20240502 20:46:52.575693 9283 raft_consensus.cc:3032] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.575745 9283 raft_consensus.cc:729] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.575897 9283 consensus_queue.cc:260] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.576058 9283 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.577041 9283 raft_consensus.cc:2454] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 2.
I20240502 20:46:52.578611 9283 raft_consensus.cc:1261] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 20:46:52.579502 9557 raft_consensus.cc:1261] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 20:46:52.581471 9355 leader_election.cc:302] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:52.582160 11628 raft_consensus.cc:2781] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.582237 11628 raft_consensus.cc:686] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.582326 11628 consensus_queue.cc:237] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.583592 11628 consensus_queue.cc:1035] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.584043 9144 catalog_manager.cc:5484] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 1 to 2, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 2 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
W20240502 20:46:52.587316 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51646: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:52.588224 10462 consensus_queue.cc:1035] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.588318 10462 consensus_queue.cc:1225] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Peer 190c1d3968d949bbbaa27a51a2b91d8e log is divergent from this leader: its last log entry 2.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20240502 20:46:52.590338 9556 pending_rounds.cc:76] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e: Aborting all ops after (but not including) 3
I20240502 20:46:52.590394 9556 pending_rounds.cc:97] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e: Aborting uncommitted NO_OP operation due to leader change: 2.4
I20240502 20:46:52.590425 9556 raft_consensus.cc:2866] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 20:46:52.591189 9557 raft_consensus.cc:1261] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 3 index: 9. (index mismatch)
I20240502 20:46:52.592028 9557 raft_consensus.cc:1261] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 3 index: 10. (index mismatch)
I20240502 20:46:52.592129 9687 raft_consensus.cc:1261] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 3 index: 10. (index mismatch)
I20240502 20:46:52.592975 11639 consensus_queue.cc:1035] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 20:46:52.593364 11639 consensus_queue.cc:1035] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 20:46:52.595495 11639 consensus_queue.cc:1035] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 20:46:52.598335 9284 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.599366 9284 raft_consensus.cc:1261] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 3 index: 9. (index mismatch)
I20240502 20:46:52.599836 11539 mvcc.cc:205] Tried to move back new op lower bound from 7023340800358195200 to 7023340800184639488. Current Snapshot: MvccSnapshot[applied={T|T < 7023340800358195200}]
W20240502 20:46:52.611469 9248 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59764: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:52.616801 9557 raft_consensus.cc:1261] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:52.620471 9283 raft_consensus.cc:1261] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:52.624841 11628 consensus_queue.cc:1035] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 20:46:52.626775 9284 raft_consensus.cc:1261] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 20:46:52.631894 11585 consensus_queue.cc:1035] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.632952 9685 raft_consensus.cc:1261] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 20:46:52.636292 9685 raft_consensus.cc:1261] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:52.636451 11628 consensus_queue.cc:1035] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:52.636945 9556 raft_consensus.cc:1261] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 20:46:52.645635 11639 consensus_queue.cc:1035] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:52.647797 11628 consensus_queue.cc:1035] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.655947 11628 consensus_queue.cc:1035] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 20:46:52.656477 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51680: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:52.656638 11628 consensus_queue.cc:1035] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 20:46:52.671769 9647 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40120: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:52.682179 9689 raft_consensus.cc:1261] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 7. (index mismatch)
I20240502 20:46:52.682966 9554 raft_consensus.cc:1261] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 7. (index mismatch)
I20240502 20:46:52.702741 11628 consensus_queue.cc:1035] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 20:46:52.705929 9687 raft_consensus.cc:1261] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 20:46:52.706424 11577 consensus_queue.cc:1035] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.707506 11585 consensus_queue.cc:1035] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.714980 11098 heartbeater.cc:503] Master 127.22.45.126:41377 was elected leader, sending a full tablet report...
I20240502 20:46:52.723488 9289 raft_consensus.cc:1261] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 20:46:52.725721 9289 raft_consensus.cc:1261] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 20:46:52.725960 9556 raft_consensus.cc:1261] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 20:46:52.726579 9689 raft_consensus.cc:1261] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 20:46:52.729480 9556 raft_consensus.cc:1261] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 20:46:52.729991 11639 consensus_queue.cc:1035] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.730716 11585 consensus_queue.cc:1035] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.731498 9556 raft_consensus.cc:1261] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 4. (term mismatch)
I20240502 20:46:52.731561 9556 pending_rounds.cc:76] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e: Aborting all ops after (but not including) 3
I20240502 20:46:52.731586 9556 pending_rounds.cc:97] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e: Aborting uncommitted NO_OP operation due to leader change: 2.4
I20240502 20:46:52.731617 9556 raft_consensus.cc:2866] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 20:46:52.731595 10462 consensus_queue.cc:1035] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.731669 10462 consensus_queue.cc:1225] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Peer 190c1d3968d949bbbaa27a51a2b91d8e log is divergent from this leader: its last log entry 2.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20240502 20:46:52.731799 11710 consensus_queue.cc:1035] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.731964 11710 consensus_queue.cc:1225] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Peer 190c1d3968d949bbbaa27a51a2b91d8e log is divergent from this leader: its last log entry 2.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20240502 20:46:52.741504 11577 consensus_queue.cc:1035] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.745110 9285 raft_consensus.cc:1261] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20240502 20:46:52.751709 11714 consensus_queue.cc:1035] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.766561 9283 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:52.766713 9283 raft_consensus.cc:482] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.766786 9283 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.767823 9283 raft_consensus.cc:504] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.768210 9283 leader_election.cc:288] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 2 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:52.768887 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.769008 9687 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.770062 9687 raft_consensus.cc:2454] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 2.
I20240502 20:46:52.770510 9223 leader_election.cc:302] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:52.770763 11667 raft_consensus.cc:2781] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.770852 11667 raft_consensus.cc:686] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.770926 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.771034 9423 raft_consensus.cc:3032] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.770987 11667 consensus_queue.cc:237] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.771080 9423 raft_consensus.cc:729] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.771185 9423 consensus_queue.cc:260] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.771286 9423 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.771627 9288 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:52.771751 9288 raft_consensus.cc:482] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.771804 9288 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.772286 9423 raft_consensus.cc:2396] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 2 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20240502 20:46:52.772418 11639 consensus_queue.cc:1035] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.772716 9288 raft_consensus.cc:504] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.773059 9288 leader_election.cc:288] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 3 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:52.776996 9557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.777112 9557 raft_consensus.cc:3032] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:52.777148 9557 raft_consensus.cc:729] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.777184 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.777252 9557 consensus_queue.cc:260] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.777292 9686 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.777387 9557 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.778239 9686 raft_consensus.cc:2454] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 3.
I20240502 20:46:52.778331 9557 raft_consensus.cc:2454] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 3.
I20240502 20:46:52.778823 9223 leader_election.cc:302] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:52.779167 10895 raft_consensus.cc:2781] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Leader election won for term 3
I20240502 20:46:52.779258 10895 raft_consensus.cc:686] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.779397 10895 consensus_queue.cc:237] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.780467 9289 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:52.780573 9289 raft_consensus.cc:482] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.780635 9289 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.781484 9289 raft_consensus.cc:504] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.781770 9289 leader_election.cc:288] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 4 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:52.784210 9557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.784329 9557 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.785205 9557 raft_consensus.cc:2454] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 4.
I20240502 20:46:52.789062 9223 leader_election.cc:302] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:52.789064 9289 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:52.789203 9289 raft_consensus.cc:482] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.789243 11667 raft_consensus.cc:2781] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Leader election won for term 4
I20240502 20:46:52.789261 9289 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.789336 11667 raft_consensus.cc:686] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.789451 11667 consensus_queue.cc:237] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.790580 9289 raft_consensus.cc:504] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.790897 9289 leader_election.cc:288] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 3 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:52.791306 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.791430 9688 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.791563 9557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.791666 9557 raft_consensus.cc:3032] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:52.791708 9557 raft_consensus.cc:729] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.791847 9557 consensus_queue.cc:260] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.792007 9557 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.792367 9688 raft_consensus.cc:2454] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 3.
I20240502 20:46:52.792836 9557 raft_consensus.cc:2454] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 3.
I20240502 20:46:52.793129 9223 leader_election.cc:302] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:52.793450 9289 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:52.793552 10895 raft_consensus.cc:2781] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Leader election won for term 3
I20240502 20:46:52.793639 10895 raft_consensus.cc:686] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.793757 10895 consensus_queue.cc:237] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.795929 9555 pending_rounds.cc:76] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e: Aborting all ops after (but not including) 3
I20240502 20:46:52.795977 9555 pending_rounds.cc:97] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e: Aborting uncommitted NO_OP operation due to leader change: 2.4
I20240502 20:46:52.796005 9555 raft_consensus.cc:2866] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 20:46:52.796241 9551 raft_consensus.cc:1226] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Rejecting Update request from peer 88a19090795d47c2a1ea1dd979fb62bc for earlier term 3. Current term is 4. Ops: [3.4-3.4]
I20240502 20:46:52.796263 9553 pending_rounds.cc:76] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e: Aborting all ops after (but not including) 3
I20240502 20:46:52.796310 9553 pending_rounds.cc:97] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e: Aborting uncommitted NO_OP operation due to leader change: 2.4
I20240502 20:46:52.796336 9553 raft_consensus.cc:2866] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20240502 20:46:52.798154 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51672: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:52.799759 11564 mvcc.cc:205] Tried to move back new op lower bound from 7023340800717742080 to 7023340800265764864. Current Snapshot: MvccSnapshot[applied={T|T < 7023340800717742080}]
I20240502 20:46:52.793555 9289 raft_consensus.cc:482] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.801862 9289 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.802688 9289 raft_consensus.cc:504] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.803371 9289 leader_election.cc:288] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 2 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:52.803912 9288 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:52.804021 9288 raft_consensus.cc:482] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.804082 9288 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.804208 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.804316 9555 raft_consensus.cc:3032] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.804359 9555 raft_consensus.cc:729] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.804472 9555 consensus_queue.cc:260] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.804602 9555 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.804895 9288 raft_consensus.cc:504] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.805189 9288 leader_election.cc:288] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 4 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:52.805512 9555 raft_consensus.cc:2454] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 2.
I20240502 20:46:52.805673 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 9 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.805780 9553 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.806059 9223 leader_election.cc:302] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:52.806222 11667 raft_consensus.cc:2781] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.806304 11667 raft_consensus.cc:686] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.806417 11667 consensus_queue.cc:237] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.806651 9553 raft_consensus.cc:2454] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 4.
I20240502 20:46:52.806797 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.806901 9688 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.807317 9223 leader_election.cc:302] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:52.807806 11667 raft_consensus.cc:2781] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Leader election won for term 4
I20240502 20:46:52.807891 11667 raft_consensus.cc:686] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.808009 11667 consensus_queue.cc:237] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 3.9, Last appended by leader: 9, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.808739 9283 raft_consensus.cc:1226] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 LEADER]: Rejecting Update request from peer 190c1d3968d949bbbaa27a51a2b91d8e for earlier term 1. Current term is 2. Ops: []
I20240502 20:46:52.809144 9288 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:52.809245 9288 raft_consensus.cc:482] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.809305 9288 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.810055 9288 raft_consensus.cc:504] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.810389 9288 leader_election.cc:288] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 3 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:52.811004 9288 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:52.811115 9288 raft_consensus.cc:482] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.811175 9288 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.811712 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.811820 9687 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.811923 9288 raft_consensus.cc:504] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.812220 9288 leader_election.cc:288] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 4 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:52.815627 11715 consensus_queue.cc:1046] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20240502 20:46:52.815833 11715 raft_consensus.cc:3032] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Stepping down as leader of term 3
I20240502 20:46:52.815883 11715 raft_consensus.cc:729] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.816022 11715 consensus_queue.cc:260] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.816161 11715 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.816476 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.816574 9553 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.817327 9553 raft_consensus.cc:2454] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 4.
I20240502 20:46:52.817792 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.818297 9685 raft_consensus.cc:1261] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 7. (index mismatch)
I20240502 20:46:52.819108 9688 raft_consensus.cc:2454] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 2.
I20240502 20:46:52.819914 9289 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3315525f1849462fbf49a285f424ea51"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:52.819994 9553 raft_consensus.cc:1261] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 7. (index mismatch)
I20240502 20:46:52.820022 9289 raft_consensus.cc:482] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.820073 9289 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.820853 9289 raft_consensus.cc:504] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.821149 9289 leader_election.cc:288] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 4 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:52.821378 10895 consensus_queue.cc:1035] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 20:46:52.821637 9223 leader_election.cc:302] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:52.821705 9423 raft_consensus.cc:2454] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 4.
I20240502 20:46:52.821959 10895 raft_consensus.cc:2781] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Leader election won for term 4
I20240502 20:46:52.821978 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 9 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.822036 10895 raft_consensus.cc:686] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.822062 9422 raft_consensus.cc:3032] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Stepping down as leader of term 3
I20240502 20:46:52.822094 9422 raft_consensus.cc:729] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.822144 10895 consensus_queue.cc:237] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.822186 9422 consensus_queue.cc:260] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 3.10, Last appended by leader: 10, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.822269 9422 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.822284 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3315525f1849462fbf49a285f424ea51" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.822372 9553 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.823130 9422 raft_consensus.cc:2396] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 4 because replica has last-logged OpId of term: 3 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 9.
I20240502 20:46:52.823125 9553 raft_consensus.cc:2454] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 4.
I20240502 20:46:52.823127 9687 raft_consensus.cc:2454] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 3.
I20240502 20:46:52.823637 9223 leader_election.cc:302] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:52.823668 11667 consensus_queue.cc:1035] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 20:46:52.823912 9223 leader_election.cc:302] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:52.823926 11667 raft_consensus.cc:2781] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Leader election won for term 4
I20240502 20:46:52.824159 11667 raft_consensus.cc:686] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.824285 10895 raft_consensus.cc:2781] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Leader election won for term 3
I20240502 20:46:52.824293 11667 consensus_queue.cc:237] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 3.12, Last appended by leader: 12, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.824453 10895 raft_consensus.cc:686] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.824566 10895 consensus_queue.cc:237] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 6, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.825105 9289 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fe22224142984d69a2def286584b9aaa"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:52.825203 9289 raft_consensus.cc:482] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.825254 9289 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.826418 9136 catalog_manager.cc:5484] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 2 to 3, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 3 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:52.829411 9289 raft_consensus.cc:504] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.829728 9289 leader_election.cc:288] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 3 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:52.830554 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.830658 9422 raft_consensus.cc:3032] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:52.830708 9422 raft_consensus.cc:729] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.830838 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe22224142984d69a2def286584b9aaa" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 18 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.830844 9422 consensus_queue.cc:260] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 6, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.830936 9688 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.830993 9422 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.831722 9688 raft_consensus.cc:2454] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 3.
I20240502 20:46:52.831722 9422 raft_consensus.cc:2454] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 3.
I20240502 20:46:52.832099 9223 leader_election.cc:302] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:52.832374 11667 raft_consensus.cc:2781] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Leader election won for term 3
I20240502 20:46:52.832459 11667 raft_consensus.cc:686] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.832579 11667 consensus_queue.cc:237] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 2.18, Last appended by leader: 18, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.833657 9553 raft_consensus.cc:1226] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Rejecting Update request from peer 88a19090795d47c2a1ea1dd979fb62bc for earlier term 3. Current term is 4. Ops: [3.12-3.12]
I20240502 20:46:52.835883 9289 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:52.835983 9289 raft_consensus.cc:482] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.836042 9289 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.836695 9289 raft_consensus.cc:504] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.837010 9289 leader_election.cc:288] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 3 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:52.837359 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.837462 9422 raft_consensus.cc:3032] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Stepping down as leader of term 3
I20240502 20:46:52.837507 9422 raft_consensus.cc:729] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.837615 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 12 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.837626 9422 consensus_queue.cc:260] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.837714 9422 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.837796 9688 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.838670 9422 raft_consensus.cc:2396] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20240502 20:46:52.839018 9688 raft_consensus.cc:2396] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 3 because replica has last-logged OpId of term: 2 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 12.
I20240502 20:46:52.839437 9289 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:52.839583 9289 raft_consensus.cc:482] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.839730 9289 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.840051 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3315525f1849462fbf49a285f424ea51" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.840150 9422 raft_consensus.cc:3032] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Stepping down as leader of term 3
I20240502 20:46:52.840190 9422 raft_consensus.cc:729] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.840323 9422 consensus_queue.cc:260] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 3.12, Last appended by leader: 12, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.840431 9422 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.840801 9289 raft_consensus.cc:504] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.841163 9422 raft_consensus.cc:2454] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 4.
I20240502 20:46:52.841447 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.841555 9553 raft_consensus.cc:3032] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.841605 9553 raft_consensus.cc:729] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.841725 9553 consensus_queue.cc:260] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.841893 9553 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.841929 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.842095 9688 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.842527 9289 leader_election.cc:288] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 2 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:52.842710 9553 raft_consensus.cc:2454] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 2.
I20240502 20:46:52.843117 9688 raft_consensus.cc:2454] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 2.
I20240502 20:46:52.843132 9223 leader_election.cc:302] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:52.843276 10895 raft_consensus.cc:2781] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.843372 9289 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:52.843381 10895 raft_consensus.cc:686] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.843528 9289 raft_consensus.cc:482] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.843578 9289 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.843582 10895 consensus_queue.cc:237] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.844292 9289 raft_consensus.cc:504] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.844374 9553 raft_consensus.cc:1226] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Rejecting Update request from peer 88a19090795d47c2a1ea1dd979fb62bc for earlier term 3. Current term is 4. Ops: [3.10-3.10]
I20240502 20:46:52.844708 9289 leader_election.cc:288] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 3 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:52.844949 9553 raft_consensus.cc:1226] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Rejecting Update request from peer 88a19090795d47c2a1ea1dd979fb62bc for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20240502 20:46:52.845126 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.845213 9555 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.846060 9555 raft_consensus.cc:2454] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 3.
I20240502 20:46:52.846518 9223 leader_election.cc:302] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:52.846920 10895 raft_consensus.cc:2781] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Leader election won for term 3
I20240502 20:46:52.847013 10895 raft_consensus.cc:686] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.847065 9289 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:52.847123 10895 consensus_queue.cc:237] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.847177 9289 raft_consensus.cc:482] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.847229 9289 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.847946 9289 raft_consensus.cc:504] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.848062 9284 raft_consensus.cc:1226] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 LEADER]: Rejecting Update request from peer 88a19090795d47c2a1ea1dd979fb62bc for earlier term 3. Current term is 4. Ops: []
I20240502 20:46:52.848241 9289 leader_election.cc:288] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 2 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:52.848641 9687 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.848958 9685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.849057 9685 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.849527 9687 raft_consensus.cc:1261] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 20:46:52.849766 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.849872 9553 raft_consensus.cc:3032] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.849876 9685 raft_consensus.cc:2454] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 2.
I20240502 20:46:52.849910 9553 raft_consensus.cc:729] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.850025 9553 consensus_queue.cc:260] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 1.15, Last appended by leader: 15, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.850154 9553 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.850230 9223 leader_election.cc:302] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:52.850380 10895 raft_consensus.cc:2781] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.850445 10895 raft_consensus.cc:686] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.850618 10895 consensus_queue.cc:237] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 1.15, Last appended by leader: 15, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.850984 9289 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:52.851038 9553 raft_consensus.cc:2454] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 2.
I20240502 20:46:52.851090 9289 raft_consensus.cc:482] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.851153 9289 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.851862 9289 raft_consensus.cc:504] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.852170 9289 leader_election.cc:288] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 4 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:52.852833 9284 raft_consensus.cc:1226] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 LEADER]: Rejecting Update request from peer 88a19090795d47c2a1ea1dd979fb62bc for earlier term 3. Current term is 4. Ops: []
I20240502 20:46:52.853193 9685 raft_consensus.cc:1261] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20240502 20:46:52.855255 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.855454 9553 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.855670 9289 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d92bfee8fabd46a28ffcb7e319879406"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:52.855775 9289 raft_consensus.cc:482] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.855832 9289 raft_consensus.cc:3037] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.856375 9553 raft_consensus.cc:2396] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 4 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 20:46:52.856622 9289 raft_consensus.cc:504] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.857003 9289 leader_election.cc:288] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 2 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:52.857239 9688 raft_consensus.cc:1226] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Rejecting Update request from peer 88a19090795d47c2a1ea1dd979fb62bc for earlier term 2. Current term is 3. Ops: [2.19-2.19]
I20240502 20:46:52.857340 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d92bfee8fabd46a28ffcb7e319879406" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.857424 9687 raft_consensus.cc:3037] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.857589 9284 raft_consensus.cc:1226] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 LEADER]: Rejecting Update request from peer 88a19090795d47c2a1ea1dd979fb62bc for earlier term 3. Current term is 4. Ops: []
I20240502 20:46:52.858206 9687 raft_consensus.cc:2454] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 2.
I20240502 20:46:52.859104 9223 leader_election.cc:302] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:52.859256 11667 raft_consensus.cc:2781] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Leader election won for term 2
I20240502 20:46:52.859333 11667 raft_consensus.cc:686] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.859445 11667 consensus_queue.cc:237] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.859817 9284 raft_consensus.cc:1226] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Rejecting Update request from peer 88a19090795d47c2a1ea1dd979fb62bc for earlier term 2. Current term is 3. Ops: [2.13-2.13]
I20240502 20:46:52.859884 9289 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:52.859961 9289 raft_consensus.cc:482] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.860005 9289 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.860394 9687 raft_consensus.cc:1226] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Rejecting Update request from peer 88a19090795d47c2a1ea1dd979fb62bc for earlier term 2. Current term is 3. Ops: []
I20240502 20:46:52.860702 9289 raft_consensus.cc:504] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.860796 9284 raft_consensus.cc:1226] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Rejecting Update request from peer 88a19090795d47c2a1ea1dd979fb62bc for earlier term 2. Current term is 3. Ops: [2.19-2.19]
I20240502 20:46:52.860976 9289 leader_election.cc:288] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 3 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:52.861464 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.861549 9687 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.861909 9144 catalog_manager.cc:5484] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 3 to 4, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 4 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:52.862151 9144 catalog_manager.cc:5484] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 1 to 2, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 2 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:52.862311 9687 raft_consensus.cc:2454] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 3.
I20240502 20:46:52.862327 9144 catalog_manager.cc:5484] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 3 to 4, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 4 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:52.862504 9144 catalog_manager.cc:5484] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 1 to 2, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 2 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:52.862718 9223 leader_election.cc:302] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:52.862859 11667 raft_consensus.cc:2781] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Leader election won for term 3
I20240502 20:46:52.862679 9144 catalog_manager.cc:5484] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 2 to 3, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 3 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:52.863081 11667 raft_consensus.cc:686] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.863210 11667 consensus_queue.cc:237] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.863840 9289 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:52.863945 9289 raft_consensus.cc:482] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.864004 9289 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.864924 9289 raft_consensus.cc:504] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.865000 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe22224142984d69a2def286584b9aaa" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 18 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.865099 9422 raft_consensus.cc:3032] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:52.865145 9422 raft_consensus.cc:729] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.865237 9289 leader_election.cc:288] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 3 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:52.865265 9422 consensus_queue.cc:260] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 2.19, Last appended by leader: 19, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.865376 9422 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.865985 9289 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "20698b2531674eefb9df3125a86653ba"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:52.866088 9289 raft_consensus.cc:482] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.866149 9289 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.866348 9422 raft_consensus.cc:2396] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 3 because replica has last-logged OpId of term: 2 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 18.
I20240502 20:46:52.866945 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.867034 9289 raft_consensus.cc:504] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.867100 9688 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.867354 9289 leader_election.cc:288] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 4 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:52.867774 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20698b2531674eefb9df3125a86653ba" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 14 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.867877 9555 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.868096 9688 raft_consensus.cc:2454] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 3.
I20240502 20:46:52.868161 9289 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:52.868259 9289 raft_consensus.cc:482] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.868363 9289 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.868430 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.868523 9553 raft_consensus.cc:3032] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:52.868567 9553 raft_consensus.cc:729] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.868717 9553 consensus_queue.cc:260] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 6, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.868888 9553 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.869130 9555 raft_consensus.cc:2396] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 4 because replica has last-logged OpId of term: 3 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 14.
I20240502 20:46:52.869107 9289 raft_consensus.cc:504] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.869632 9289 leader_election.cc:288] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 4 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:52.869694 9553 raft_consensus.cc:2454] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 3.
I20240502 20:46:52.870043 9223 leader_election.cc:302] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:52.870306 10895 raft_consensus.cc:2781] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Leader election won for term 3
I20240502 20:46:52.870414 10895 raft_consensus.cc:686] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.870658 10895 consensus_queue.cc:237] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 6, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.871359 9289 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:52.871496 9289 raft_consensus.cc:482] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.871577 9289 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Advancing to term 3
W20240502 20:46:52.877843 9382 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55522: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:52.877858 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.877952 9553 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.878707 9553 raft_consensus.cc:2454] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 4.
I20240502 20:46:52.877839 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 12 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.879017 9422 raft_consensus.cc:3032] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:52.879060 9422 raft_consensus.cc:729] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.879119 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.879182 9422 consensus_queue.cc:260] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 2.13, Last appended by leader: 13, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.879231 9423 raft_consensus.cc:3032] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:52.879284 9423 raft_consensus.cc:729] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.879405 9423 consensus_queue.cc:260] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.879278 9422 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.880314 11715 consensus_queue.cc:1035] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:52.880523 9422 raft_consensus.cc:2396] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 3 because replica has last-logged OpId of term: 2 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 12.
I20240502 20:46:52.880744 11715 consensus_queue.cc:1035] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.880764 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.880889 9420 raft_consensus.cc:3032] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:52.880929 9420 raft_consensus.cc:729] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.881068 9420 consensus_queue.cc:260] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.881186 9420 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.882115 9420 raft_consensus.cc:2396] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20240502 20:46:52.882295 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d92bfee8fabd46a28ffcb7e319879406" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.882391 9421 raft_consensus.cc:3032] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 LEADER]: Stepping down as leader of term 1
I20240502 20:46:52.882541 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.882609 9424 raft_consensus.cc:3032] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Stepping down as leader of term 3
I20240502 20:46:52.882634 9424 raft_consensus.cc:729] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.882736 9424 consensus_queue.cc:260] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.882831 9424 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.882874 9223 leader_election.cc:302] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:52.883011 11667 raft_consensus.cc:2781] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Leader election won for term 4
I20240502 20:46:52.883083 11667 raft_consensus.cc:686] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.883190 11667 consensus_queue.cc:237] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 3.12, Last appended by leader: 12, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.883647 9424 raft_consensus.cc:2396] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 20:46:52.884065 9289 raft_consensus.cc:504] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.884377 9289 leader_election.cc:288] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 3 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:52.882432 9421 raft_consensus.cc:729] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.884886 9421 consensus_queue.cc:260] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.885020 9421 raft_consensus.cc:3037] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 1 FOLLOWER]: Advancing to term 2
I20240502 20:46:52.885677 9289 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:52.885771 9289 raft_consensus.cc:482] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.885819 9289 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.885813 9421 raft_consensus.cc:2454] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 2.
I20240502 20:46:52.886313 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.886413 9688 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.886569 9423 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.886629 9289 raft_consensus.cc:504] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.886907 9289 leader_election.cc:288] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 3 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:52.887378 9423 raft_consensus.cc:2454] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 3.
I20240502 20:46:52.887480 9688 raft_consensus.cc:2454] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 3.
I20240502 20:46:52.887859 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:52.888136 9687 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.888216 9685 raft_consensus.cc:1226] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Rejecting Update request from peer 190c1d3968d949bbbaa27a51a2b91d8e for earlier term 2. Current term is 3. Ops: [2.15-2.15]
W20240502 20:46:52.888295 9382 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55574: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:52.888692 10554 consensus_queue.cc:1046] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: INVALID_TERM, Last received: 2.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 20:46:52.888886 10554 raft_consensus.cc:3032] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:52.888940 10554 raft_consensus.cc:729] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:52.889055 10554 consensus_queue.cc:260] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.15, Last appended by leader: 15, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.889195 10554 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Advancing to term 3
W20240502 20:46:52.889279 9382 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:52.889662 11232 heartbeater.cc:503] Master 127.22.45.126:41377 was elected leader, sending a full tablet report...
I20240502 20:46:52.890758 9687 raft_consensus.cc:2454] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 3.
I20240502 20:46:52.891028 9223 leader_election.cc:302] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:52.891166 11667 raft_consensus.cc:2781] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Leader election won for term 3
I20240502 20:46:52.891222 11667 raft_consensus.cc:686] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.891317 11667 consensus_queue.cc:237] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.892123 9223 leader_election.cc:302] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:52.892270 10895 raft_consensus.cc:2781] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Leader election won for term 3
I20240502 20:46:52.892349 10895 raft_consensus.cc:686] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.892465 10895 consensus_queue.cc:237] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.9, Last appended by leader: 9, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.893376 9284 raft_consensus.cc:1226] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 LEADER]: Rejecting Update request from peer 190c1d3968d949bbbaa27a51a2b91d8e for earlier term 1. Current term is 2. Ops: []
I20240502 20:46:52.893746 9222 leader_election.cc:302] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f; no voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:52.893908 11667 raft_consensus.cc:2726] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20240502 20:46:52.894029 9222 leader_election.cc:302] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:52.894140 11667 raft_consensus.cc:2726] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20240502 20:46:52.894136 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51672: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:52.894806 9284 raft_consensus.cc:1226] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Rejecting Update request from peer 88a19090795d47c2a1ea1dd979fb62bc for earlier term 3. Current term is 4. Ops: [3.15-3.15]
I20240502 20:46:52.894894 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20698b2531674eefb9df3125a86653ba" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 14 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.894979 9423 raft_consensus.cc:3032] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Stepping down as leader of term 3
I20240502 20:46:52.895013 9423 raft_consensus.cc:729] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.895154 9284 raft_consensus.cc:1226] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 LEADER]: Rejecting Update request from peer 88a19090795d47c2a1ea1dd979fb62bc for earlier term 1. Current term is 2. Ops: []
I20240502 20:46:52.895119 9423 consensus_queue.cc:260] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 3.15, Last appended by leader: 15, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.895216 9423 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.895484 9284 raft_consensus.cc:1226] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Rejecting Update request from peer 88a19090795d47c2a1ea1dd979fb62bc for earlier term 2. Current term is 3. Ops: []
I20240502 20:46:52.896008 9284 raft_consensus.cc:1226] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Rejecting Update request from peer 190c1d3968d949bbbaa27a51a2b91d8e for earlier term 2. Current term is 3. Ops: [2.15-2.15]
I20240502 20:46:52.896111 9423 raft_consensus.cc:2396] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 4 because replica has last-logged OpId of term: 3 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 14.
I20240502 20:46:52.896314 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.896405 9421 raft_consensus.cc:3032] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Stepping down as leader of term 3
I20240502 20:46:52.896440 9421 raft_consensus.cc:729] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.896543 9421 consensus_queue.cc:260] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 3.12, Last appended by leader: 12, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.896689 9421 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.896641 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.896826 9553 raft_consensus.cc:2396] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 3 because replica has last-logged OpId of term: 2 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 14.
I20240502 20:46:52.897485 9421 raft_consensus.cc:2454] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 4.
I20240502 20:46:52.897647 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.897729 9424 raft_consensus.cc:3032] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:52.897768 9424 raft_consensus.cc:729] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.897876 9424 consensus_queue.cc:260] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.10, Last appended by leader: 10, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.897969 9424 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.898140 9554 raft_consensus.cc:1226] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Rejecting Update request from peer 88a19090795d47c2a1ea1dd979fb62bc for earlier term 3. Current term is 4. Ops: []
I20240502 20:46:52.898802 9424 raft_consensus.cc:2396] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 3 because replica has last-logged OpId of term: 2 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 9.
I20240502 20:46:52.899628 9144 catalog_manager.cc:5484] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 1 to 2, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 2 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:52.899888 9144 catalog_manager.cc:5484] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 2 to 3, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 3 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:52.900072 9144 catalog_manager.cc:5484] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 1 to 2, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 2 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:52.900230 9144 catalog_manager.cc:5484] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 2 to 3, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 3 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:52.900337 9144 catalog_manager.cc:5484] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 1 to 2, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 2 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:52.900466 9144 catalog_manager.cc:5484] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 3 to 4, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 4 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:52.900599 9144 catalog_manager.cc:5484] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 2 to 3, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 3 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:52.900740 9144 catalog_manager.cc:5484] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 3 to 4, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 4 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:52.900909 9144 catalog_manager.cc:5484] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 2 to 3, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 3 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:52.901690 9689 raft_consensus.cc:1226] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Rejecting Update request from peer 88a19090795d47c2a1ea1dd979fb62bc for earlier term 2. Current term is 3. Ops: [2.10-2.10]
I20240502 20:46:52.901077 9144 catalog_manager.cc:5484] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 3 to 4, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 4 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
W20240502 20:46:52.903362 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51610: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:52.903414 9144 catalog_manager.cc:5484] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 2 to 3, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 3 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:52.903564 9144 catalog_manager.cc:5484] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 2 to 3, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 3 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:52.903666 9555 raft_consensus.cc:1261] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 7. (index mismatch)
I20240502 20:46:52.903702 9144 catalog_manager.cc:5484] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 2 to 3, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 3 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:52.904043 11725 consensus_queue.cc:1035] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20240502 20:46:52.906103 9647 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:52.906610 11558 mvcc.cc:205] Tried to move back new op lower bound from 7023340800462680064 to 7023340799903367168. Current Snapshot: MvccSnapshot[applied={T|T < 7023340800462680064 or (T in {7023340800462680064})}]
I20240502 20:46:52.912370 9553 raft_consensus.cc:1261] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 20:46:52.912760 11711 consensus_queue.cc:1035] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 20:46:52.912931 9689 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:52.913048 9689 raft_consensus.cc:482] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.913105 9689 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.914026 9689 raft_consensus.cc:504] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.914191 9688 raft_consensus.cc:1261] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20240502 20:46:52.914351 9689 leader_election.cc:288] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 4 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:52.914750 9284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.914862 9284 raft_consensus.cc:3032] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Stepping down as leader of term 3
I20240502 20:46:52.914902 9284 raft_consensus.cc:729] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.915017 9284 consensus_queue.cc:260] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.915132 9284 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Advancing to term 4
W20240502 20:46:52.915437 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51640: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:52.916056 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.916098 9284 raft_consensus.cc:2396] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20240502 20:46:52.916141 9555 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.916668 9554 raft_consensus.cc:1261] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 3 index: 11. Preceding OpId from leader: term: 4 index: 14. (index mismatch)
I20240502 20:46:52.916930 9689 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:52.916980 9555 raft_consensus.cc:2454] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 4.
I20240502 20:46:52.917026 9689 raft_consensus.cc:482] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.917071 9689 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.917443 9623 leader_election.cc:302] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters: 4b3cb6ad00954dc5a1270dc392870b4f
I20240502 20:46:52.917646 10960 raft_consensus.cc:2781] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 20:46:52.917704 10960 raft_consensus.cc:686] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:52.917737 10895 consensus_queue.cc:1035] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 20:46:52.917815 9689 raft_consensus.cc:504] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.917989 10895 consensus_queue.cc:1035] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 20:46:52.918123 9689 leader_election.cc:288] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 3 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:52.917806 10960 consensus_queue.cc:237] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.919579 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.919675 9424 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.920444 9284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.920518 9424 raft_consensus.cc:2396] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 3 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20240502 20:46:52.920529 9284 raft_consensus.cc:3032] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:52.920562 9284 raft_consensus.cc:729] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.920665 9284 consensus_queue.cc:260] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.920791 9284 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.921231 9689 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:52.921356 9689 raft_consensus.cc:482] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.921406 9689 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Advancing to term 4
W20240502 20:46:52.921571 9647 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40106: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:52.921591 9284 raft_consensus.cc:2396] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20240502 20:46:52.921842 9621 leader_election.cc:302] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:52.922040 10960 raft_consensus.cc:2726] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20240502 20:46:52.922082 9689 raft_consensus.cc:504] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.922576 9136 catalog_manager.cc:5484] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 3 to 4, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 4 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:52.923230 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.923338 9424 raft_consensus.cc:3032] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Stepping down as leader of term 3
I20240502 20:46:52.923377 9424 raft_consensus.cc:729] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.923488 9424 consensus_queue.cc:260] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 3.7, Last appended by leader: 7, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.923610 9424 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.924248 9686 raft_consensus.cc:1226] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Rejecting Update request from peer 88a19090795d47c2a1ea1dd979fb62bc for earlier term 3. Current term is 4. Ops: [3.7-3.7]
I20240502 20:46:52.924382 9424 raft_consensus.cc:2396] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 4 because replica has last-logged OpId of term: 3 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20240502 20:46:52.925384 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.925478 9553 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Advancing to term 4
W20240502 20:46:52.926446 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51610: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:52.926581 9689 leader_election.cc:288] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 4 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:52.927316 9686 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:52.927405 9686 raft_consensus.cc:482] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.927464 9686 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.928113 9686 raft_consensus.cc:504] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.928428 9686 leader_election.cc:288] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 4 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:52.928661 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 4 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.928750 9555 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.929163 9284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 4 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.929247 9284 raft_consensus.cc:3032] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Stepping down as leader of term 3
I20240502 20:46:52.929277 9284 raft_consensus.cc:729] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.929386 9284 consensus_queue.cc:260] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 3.7, Last appended by leader: 7, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.929512 9284 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.929574 9555 raft_consensus.cc:2454] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 4.
I20240502 20:46:52.929975 9623 leader_election.cc:302] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:52.930280 9284 raft_consensus.cc:2454] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 4.
I20240502 20:46:52.931018 10960 raft_consensus.cc:2781] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 20:46:52.931106 10960 raft_consensus.cc:686] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:52.931214 10960 consensus_queue.cc:237] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 3.7, Last appended by leader: 7, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.926499 9553 raft_consensus.cc:2396] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 4 because replica has last-logged OpId of term: 3 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20240502 20:46:52.932355 9623 leader_election.cc:302] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:52.932680 10960 raft_consensus.cc:2726] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 20:46:52.932734 9686 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:52.932826 9686 raft_consensus.cc:482] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.932875 9686 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.933668 9686 raft_consensus.cc:504] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.933988 9686 leader_election.cc:288] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 3 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:52.934262 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.934381 9555 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.935170 9555 raft_consensus.cc:2454] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 3.
I20240502 20:46:52.935223 9284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.935321 9284 raft_consensus.cc:3032] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:52.935359 9284 raft_consensus.cc:729] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.935482 9623 leader_election.cc:302] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:52.935472 9284 consensus_queue.cc:260] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.13, Last appended by leader: 13, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.935580 9284 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.935647 11251 raft_consensus.cc:2781] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Leader election won for term 3
W20240502 20:46:52.935657 11697 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59728: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.129s ago.
I20240502 20:46:52.935722 11251 raft_consensus.cc:686] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 3 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:52.935827 11251 consensus_queue.cc:237] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.936621 9686 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:52.936707 9686 raft_consensus.cc:482] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.936730 9284 raft_consensus.cc:2396] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 3 because replica has last-logged OpId of term: 2 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 12.
I20240502 20:46:52.936748 9686 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.937598 9686 raft_consensus.cc:504] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.938040 9686 leader_election.cc:288] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 4 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:52.938094 9136 catalog_manager.cc:5484] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 3 to 4, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 4 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:52.939410 9284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 4 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.939505 9284 raft_consensus.cc:3032] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Stepping down as leader of term 3
I20240502 20:46:52.939539 9284 raft_consensus.cc:729] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.939632 9284 consensus_queue.cc:260] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.7, Last appended by leader: 7, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.939738 9284 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.940469 9284 raft_consensus.cc:2396] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 4 because replica has last-logged OpId of term: 3 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 6.
I20240502 20:46:52.940836 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 4 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.940925 9555 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.940984 9554 raft_consensus.cc:1261] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20240502 20:46:52.941099 9686 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:52.941190 9686 raft_consensus.cc:482] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.941238 9686 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.941296 10960 consensus_queue.cc:1035] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 20:46:52.941812 9555 raft_consensus.cc:2454] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 4.
I20240502 20:46:52.941905 9686 raft_consensus.cc:504] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.942063 9623 leader_election.cc:302] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:52.942394 9686 leader_election.cc:288] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 3 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:52.942430 10960 raft_consensus.cc:2781] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 20:46:52.942514 10960 raft_consensus.cc:686] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:52.942600 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.942620 10960 consensus_queue.cc:237] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.942688 9424 raft_consensus.cc:3032] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:52.942723 9424 raft_consensus.cc:729] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.942853 9424 consensus_queue.cc:260] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.942999 9424 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.943825 9424 raft_consensus.cc:2454] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 3.
I20240502 20:46:52.944458 9622 leader_election.cc:302] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:52.944551 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.944636 9553 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.944671 10960 raft_consensus.cc:2781] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 20:46:52.944751 10960 raft_consensus.cc:686] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 3 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:52.944856 10960 consensus_queue.cc:237] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
W20240502 20:46:52.946867 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51610: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:52.946877 9144 catalog_manager.cc:5484] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 2 to 3, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 3 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:52.947886 9553 raft_consensus.cc:2454] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 3.
I20240502 20:46:52.948136 9686 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:52.948233 9686 raft_consensus.cc:482] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.948289 9686 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.948607 9136 catalog_manager.cc:5484] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 2 to 3, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 3 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:52.948799 9136 catalog_manager.cc:5484] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 3 to 4, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 4 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:52.949291 9686 raft_consensus.cc:504] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.949362 9284 raft_consensus.cc:1261] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 2 index: 13. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20240502 20:46:52.949681 10960 consensus_queue.cc:1035] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 20:46:52.949776 10960 consensus_queue.cc:1225] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Peer 4b3cb6ad00954dc5a1270dc392870b4f log is divergent from this leader: its last log entry 2.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20240502 20:46:52.949951 9686 leader_election.cc:288] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 4 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:52.950573 9688 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fe22224142984d69a2def286584b9aaa"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:52.950690 9688 raft_consensus.cc:482] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.950747 9688 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Advancing to term 4
W20240502 20:46:52.951036 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59728: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:52.951249 9286 raft_consensus.cc:1203] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Deduplicated request from leader. Original: 1.11->[1.12-3.14] Dedup: 1.12->[3.13-3.14]
I20240502 20:46:52.951373 9286 pending_rounds.cc:76] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f: Aborting all ops after (but not including) 12
I20240502 20:46:52.951402 9286 pending_rounds.cc:97] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f: Aborting uncommitted NO_OP operation due to leader change: 2.13
I20240502 20:46:52.951427 9286 raft_consensus.cc:2866] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 20:46:52.951495 9688 raft_consensus.cc:504] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.951772 9688 leader_election.cc:288] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 4 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:52.952405 9284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 4 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.952947 9688 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:52.953054 9688 raft_consensus.cc:482] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.953121 9688 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.953572 9284 raft_consensus.cc:3032] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Stepping down as leader of term 3
I20240502 20:46:52.953627 9284 raft_consensus.cc:729] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.953766 9284 consensus_queue.cc:260] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.7, Last appended by leader: 7, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.953866 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe22224142984d69a2def286584b9aaa" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 4 candidate_status { last_received { term: 2 index: 18 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.953902 9284 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.953971 9421 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.953964 9688 raft_consensus.cc:504] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.954262 9688 leader_election.cc:288] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 3 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:52.954568 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 3 candidate_status { last_received { term: 2 index: 10 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.954660 9423 raft_consensus.cc:3032] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:52.954695 9423 raft_consensus.cc:729] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.954715 9284 raft_consensus.cc:2396] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 4 because replica has last-logged OpId of term: 3 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 6.
I20240502 20:46:52.954820 9423 consensus_queue.cc:260] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 2.10, Last appended by leader: 10, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.954901 9421 raft_consensus.cc:2396] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 4 because replica has last-logged OpId of term: 2 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 18.
I20240502 20:46:52.954981 9423 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.955013 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe22224142984d69a2def286584b9aaa" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 4 candidate_status { last_received { term: 2 index: 18 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.955092 9289 raft_consensus.cc:3032] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Stepping down as leader of term 3
I20240502 20:46:52.955124 9289 raft_consensus.cc:729] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.950349 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 4 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.955253 9289 consensus_queue.cc:260] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 3.19, Last appended by leader: 19, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.955319 9424 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.955363 9289 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.955783 9423 raft_consensus.cc:2454] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 3.
I20240502 20:46:52.956171 9622 leader_election.cc:302] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:52.956184 9289 raft_consensus.cc:2396] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 4 because replica has last-logged OpId of term: 3 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 18.
I20240502 20:46:52.956207 9424 raft_consensus.cc:2454] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 4.
I20240502 20:46:52.956328 10960 raft_consensus.cc:2781] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 20:46:52.956406 10960 raft_consensus.cc:686] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 3 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:52.956511 10960 consensus_queue.cc:237] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 2.10, Last appended by leader: 10, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.956549 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 3 candidate_status { last_received { term: 2 index: 10 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.956634 9553 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.957006 9621 leader_election.cc:302] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:52.957368 9553 raft_consensus.cc:2454] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 3.
I20240502 20:46:52.957743 9622 leader_election.cc:302] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 4b3cb6ad00954dc5a1270dc392870b4f
I20240502 20:46:52.957886 11251 raft_consensus.cc:2726] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 20:46:52.957974 11251 raft_consensus.cc:2781] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 20:46:52.958037 11251 raft_consensus.cc:686] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
W20240502 20:46:52.958150 9382 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55574: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:52.958137 11251 consensus_queue.cc:237] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:52.958379 9421 raft_consensus.cc:1261] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20240502 20:46:52.958729 9423 raft_consensus.cc:1261] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 3 index: 12. Preceding OpId from leader: term: 4 index: 14. (index mismatch)
I20240502 20:46:52.958899 9222 leader_election.cc:302] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:52.959034 10895 raft_consensus.cc:2726] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 20:46:52.959290 10895 consensus_queue.cc:1035] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 20:46:52.959422 11725 consensus_queue.cc:1035] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 20:46:52.959791 9686 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "edebc185b24441da9504c78394378f4d"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:52.959892 9686 raft_consensus.cc:482] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.959940 9686 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Advancing to term 4
W20240502 20:46:52.960075 9382 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:52.960629 9686 raft_consensus.cc:504] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.960938 9686 leader_election.cc:288] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 4 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:52.961297 9555 raft_consensus.cc:1261] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 4 index: 8. (index mismatch)
I20240502 20:46:52.962330 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edebc185b24441da9504c78394378f4d" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 4 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:52.962641 9289 raft_consensus.cc:1261] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 3 index: 7. Preceding OpId from leader: term: 4 index: 8. (index mismatch)
I20240502 20:46:52.962635 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edebc185b24441da9504c78394378f4d" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 4 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.962723 9421 raft_consensus.cc:3032] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Stepping down as leader of term 3
I20240502 20:46:52.962760 9421 raft_consensus.cc:729] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:52.962966 10960 consensus_queue.cc:1035] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 20:46:52.962875 9421 consensus_queue.cc:260] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 3.12, Last appended by leader: 12, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.963186 11624 consensus_queue.cc:1035] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 20:46:52.963249 11624 consensus_queue.cc:1225] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Peer 4b3cb6ad00954dc5a1270dc392870b4f log is divergent from this leader: its last log entry 3.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20240502 20:46:52.963279 9421 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.962426 9555 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.964058 9421 raft_consensus.cc:2454] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 4.
I20240502 20:46:52.964380 9622 leader_election.cc:302] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:52.964530 10960 raft_consensus.cc:2781] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 20:46:52.964552 9555 raft_consensus.cc:2454] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 4.
I20240502 20:46:52.964643 10960 raft_consensus.cc:686] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:52.964749 10960 consensus_queue.cc:237] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 3.12, Last appended by leader: 12, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:52.966040 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:52.966135 9687 raft_consensus.cc:482] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.966182 9687 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.967012 9687 raft_consensus.cc:504] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:52.967350 9687 leader_election.cc:288] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 4 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:52.967697 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 4 candidate_status { last_received { term: 2 index: 13 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:52.967767 9553 raft_consensus.cc:1261] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 4. (term mismatch)
I20240502 20:46:52.967808 9553 pending_rounds.cc:76] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e: Aborting all ops after (but not including) 3
I20240502 20:46:52.967798 9421 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.967828 9553 pending_rounds.cc:97] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e: Aborting uncommitted NO_OP operation due to leader change: 2.4
I20240502 20:46:52.967852 9553 raft_consensus.cc:2866] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 20:46:52.968570 11715 consensus_queue.cc:1035] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:52.968684 9424 raft_consensus.cc:1261] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 7. (index mismatch)
I20240502 20:46:52.969748 11739 consensus_queue.cc:1035] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20240502 20:46:52.973382 9647 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:52.973668 9421 raft_consensus.cc:2454] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 4.
I20240502 20:46:52.974321 9622 leader_election.cc:302] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:52.974498 10960 raft_consensus.cc:2781] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 20:46:52.974619 10960 raft_consensus.cc:686] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:52.974987 10960 consensus_queue.cc:237] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 2.13, Last appended by leader: 13, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
W20240502 20:46:52.977830 9647 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40106: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
W20240502 20:46:52.977838 9648 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40138: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:52.978650 9289 pending_rounds.cc:76] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f: Aborting all ops after (but not including) 6
I20240502 20:46:52.978693 9289 pending_rounds.cc:97] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f: Aborting uncommitted NO_OP operation due to leader change: 3.7
I20240502 20:46:52.978718 9289 raft_consensus.cc:2866] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 20:46:52.979983 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 4 candidate_status { last_received { term: 2 index: 13 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.980073 9289 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:52.980604 9144 catalog_manager.cc:5484] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 2 to 3, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 3 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:52.980931 9289 raft_consensus.cc:2454] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 4.
W20240502 20:46:52.982923 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59752: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
W20240502 20:46:52.982923 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51672: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
W20240502 20:46:52.983613 9382 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55614: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:52.983924 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:52.984009 9687 raft_consensus.cc:482] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:52.984053 9687 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.984918 9687 raft_consensus.cc:504] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.985248 9687 leader_election.cc:288] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 3 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:52.985157 9144 catalog_manager.cc:5484] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 3 to 4, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 4 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:52.985375 9144 catalog_manager.cc:5484] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 2 to 4, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 4 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:52.985553 9144 catalog_manager.cc:5484] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 3 to 4, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 4 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:52.987741 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 3 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:52.987836 9289 raft_consensus.cc:3032] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:52.987874 9289 raft_consensus.cc:729] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:52.987993 9289 consensus_queue.cc:260] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.16, Last appended by leader: 16, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:52.988117 9289 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:52.988978 9289 raft_consensus.cc:2396] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 3 because replica has last-logged OpId of term: 2 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 15.
I20240502 20:46:52.991076 9289 raft_consensus.cc:1261] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 2 index: 12. Preceding OpId from leader: term: 4 index: 15. (index mismatch)
W20240502 20:46:52.991286 9382 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55574: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:52.993194 11251 consensus_queue.cc:1035] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 20:46:52.993584 9424 raft_consensus.cc:1261] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 2 index: 13. Preceding OpId from leader: term: 4 index: 15. (index mismatch)
I20240502 20:46:52.995952 9687 raft_consensus.cc:1261] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20240502 20:46:52.996404 9424 raft_consensus.cc:1261] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20240502 20:46:52.996834 11725 consensus_queue.cc:1035] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:52.996930 11725 consensus_queue.cc:1225] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Peer 88a19090795d47c2a1ea1dd979fb62bc log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20240502 20:46:52.997339 11725 consensus_queue.cc:1035] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 20:46:52.997484 11624 consensus_queue.cc:1035] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 20:46:52.998934 9424 pending_rounds.cc:76] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc: Aborting all ops after (but not including) 4
I20240502 20:46:52.998991 9424 pending_rounds.cc:97] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20240502 20:46:52.999047 9424 raft_consensus.cc:2866] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20240502 20:46:53.000916 9648 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40138: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.002578 9554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 3 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.002683 9554 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:53.003657 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.003762 9687 raft_consensus.cc:482] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.003826 9687 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:53.003641 9554 raft_consensus.cc:2454] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 3.
I20240502 20:46:53.004673 9687 raft_consensus.cc:504] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.004987 9687 leader_election.cc:288] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 3 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.006191 9284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 3 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.006285 9284 raft_consensus.cc:3032] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:53.006321 9284 raft_consensus.cc:729] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.006434 9284 consensus_queue.cc:260] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.12, Last appended by leader: 12, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.006624 9284 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:53.007535 9284 raft_consensus.cc:2396] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 3 because replica has last-logged OpId of term: 2 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 11.
I20240502 20:46:53.007279 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d92bfee8fabd46a28ffcb7e319879406"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.007619 9687 raft_consensus.cc:482] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.007678 9687 raft_consensus.cc:3037] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:53.008474 9687 raft_consensus.cc:504] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.009186 9686 raft_consensus.cc:1226] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Rejecting Update request from peer 4b3cb6ad00954dc5a1270dc392870b4f for earlier term 2. Current term is 3. Ops: [2.15-2.15]
I20240502 20:46:53.009725 10895 consensus_queue.cc:1046] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: INVALID_TERM, Last received: 2.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 20:46:53.009918 10895 raft_consensus.cc:3032] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:53.009961 10895 raft_consensus.cc:729] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.010075 10895 consensus_queue.cc:260] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 2.15, Last appended by leader: 15, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.010102 9687 leader_election.cc:288] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 3 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.010178 10895 raft_consensus.cc:3037] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 2 FOLLOWER]: Advancing to term 3
W20240502 20:46:53.011443 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51672: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.012539 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d92bfee8fabd46a28ffcb7e319879406" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.012625 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.012709 9687 raft_consensus.cc:482] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.012754 9687 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.013633 9687 raft_consensus.cc:504] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.013937 9687 leader_election.cc:288] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 4 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.012642 9421 raft_consensus.cc:3037] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:53.015123 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 4 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.015218 9423 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.015235 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 3 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.015338 9553 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:53.016053 9423 raft_consensus.cc:2454] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 4.
I20240502 20:46:53.016202 9553 raft_consensus.cc:2454] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 3.
I20240502 20:46:53.016505 9622 leader_election.cc:302] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:53.016651 11251 raft_consensus.cc:2781] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 20:46:53.016712 11251 raft_consensus.cc:686] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.016817 11251 consensus_queue.cc:237] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.7, Last appended by leader: 7, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.017729 9424 raft_consensus.cc:1226] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Rejecting Update request from peer 4b3cb6ad00954dc5a1270dc392870b4f for earlier term 3. Current term is 4. Ops: []
I20240502 20:46:53.018148 11725 consensus_queue.cc:1046] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: INVALID_TERM, Last received: 3.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 20:46:53.018289 11725 raft_consensus.cc:3032] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Stepping down as leader of term 3
I20240502 20:46:53.018327 11725 raft_consensus.cc:729] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.018337 9421 raft_consensus.cc:2396] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 3 because replica has last-logged OpId of term: 2 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 14.
I20240502 20:46:53.018442 11725 consensus_queue.cc:260] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 3.7, Last appended by leader: 7, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.018637 11725 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.019603 9686 raft_consensus.cc:1226] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Rejecting Update request from peer 4b3cb6ad00954dc5a1270dc392870b4f for earlier term 3. Current term is 4. Ops: []
I20240502 20:46:53.019146 9623 leader_election.cc:302] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters: 4b3cb6ad00954dc5a1270dc392870b4f
I20240502 20:46:53.019933 11251 raft_consensus.cc:2781] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 20:46:53.020023 11251 raft_consensus.cc:686] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 3 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.020143 11251 consensus_queue.cc:237] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 1.15, Last appended by leader: 15, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.020643 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d92bfee8fabd46a28ffcb7e319879406" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.020772 9289 raft_consensus.cc:2396] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 3 because replica has last-logged OpId of term: 2 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 14.
I20240502 20:46:53.022250 9623 leader_election.cc:302] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters: 4b3cb6ad00954dc5a1270dc392870b4f
I20240502 20:46:53.022399 11624 raft_consensus.cc:2781] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 20:46:53.022500 11624 raft_consensus.cc:686] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 3 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.022619 11624 consensus_queue.cc:237] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.023008 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 4 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
W20240502 20:46:53.024644 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59752: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
W20240502 20:46:53.025162 9382 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55574: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
W20240502 20:46:53.027227 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59786: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.027454 9136 catalog_manager.cc:5484] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 2 to 3, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 3 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:53.027570 9621 leader_election.cc:302] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:53.027674 9136 catalog_manager.cc:5484] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 3 to 4, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 4 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:53.027863 9136 catalog_manager.cc:5484] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 2 to 3, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 3 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:53.028046 10960 raft_consensus.cc:2726] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20240502 20:46:53.029458 9686 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.029556 9686 raft_consensus.cc:482] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.029605 9686 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:53.029743 9289 raft_consensus.cc:2454] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 4.
W20240502 20:46:53.029945 9382 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:53.030439 9686 raft_consensus.cc:504] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.030790 9686 leader_election.cc:288] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 3 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:53.033224 9554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.033370 9554 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:53.033447 9687 raft_consensus.cc:1226] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Rejecting Update request from peer 88a19090795d47c2a1ea1dd979fb62bc for earlier term 2. Current term is 3. Ops: [2.8-2.8]
I20240502 20:46:53.034448 9554 raft_consensus.cc:2396] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 3 because replica has last-logged OpId of term: 2 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 7.
I20240502 20:46:53.034924 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.035017 9687 raft_consensus.cc:482] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.035058 9687 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.035940 9687 raft_consensus.cc:504] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.036257 9687 leader_election.cc:288] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 4 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
W20240502 20:46:53.036567 9382 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55614: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:53.037086 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.037180 9687 raft_consensus.cc:482] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.037227 9687 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Advancing to term 4
W20240502 20:46:53.037334 9648 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.037575 11712 consensus_queue.cc:1046] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: INVALID_TERM, Last received: 2.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 20:46:53.037770 11712 raft_consensus.cc:3032] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 LEADER]: Stepping down as leader of term 2
I20240502 20:46:53.037814 11712 raft_consensus.cc:729] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.037870 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.037968 9424 raft_consensus.cc:3032] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Stepping down as leader of term 3
I20240502 20:46:53.037935 11712 consensus_queue.cc:260] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 8, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.038005 9424 raft_consensus.cc:729] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.038085 11712 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 2 FOLLOWER]: Advancing to term 3
I20240502 20:46:53.038106 9424 consensus_queue.cc:260] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.038255 9424 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.038254 9554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.038340 9554 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.038368 9687 raft_consensus.cc:504] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.038687 9687 leader_election.cc:288] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 4 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.039045 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 4 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.039146 9289 raft_consensus.cc:3032] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Stepping down as leader of term 3
I20240502 20:46:53.039189 9289 raft_consensus.cc:729] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.039331 9424 raft_consensus.cc:2454] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 4.
I20240502 20:46:53.039328 9289 consensus_queue.cc:260] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 3.15, Last appended by leader: 15, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.039453 9289 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.039757 9554 raft_consensus.cc:2454] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 4.
I20240502 20:46:53.040321 9289 raft_consensus.cc:2396] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 4 because replica has last-logged OpId of term: 3 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 14.
I20240502 20:46:53.040469 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.040601 9421 raft_consensus.cc:2396] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 3 because replica has last-logged OpId of term: 2 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 7.
I20240502 20:46:53.040288 9622 leader_election.cc:302] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:53.041054 9622 leader_election.cc:302] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:53.041229 9554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 4 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.041321 9554 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.041389 11624 raft_consensus.cc:2781] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 20:46:53.041469 11624 raft_consensus.cc:686] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.041576 11624 consensus_queue.cc:237] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.041647 10960 raft_consensus.cc:2726] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20240502 20:46:53.043202 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59752: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.043471 9554 raft_consensus.cc:2396] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 4 because replica has last-logged OpId of term: 2 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 14.
W20240502 20:46:53.043649 9648 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40138: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.044260 9623 leader_election.cc:302] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f
I20240502 20:46:53.044178 9144 catalog_manager.cc:5484] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 3 to 4, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 4 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:53.044482 10960 raft_consensus.cc:2726] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 20:46:53.045359 11368 heartbeater.cc:503] Master 127.22.45.126:41377 was elected leader, sending a full tablet report...
I20240502 20:46:53.045524 9284 raft_consensus.cc:1261] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 3 index: 7. Preceding OpId from leader: term: 4 index: 9. (index mismatch)
I20240502 20:46:53.046180 9421 raft_consensus.cc:1261] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 3 index: 7. Preceding OpId from leader: term: 4 index: 9. (index mismatch)
I20240502 20:46:53.046784 11758 consensus_queue.cc:1035] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 20:46:53.046941 11251 consensus_queue.cc:1035] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240502 20:46:53.048493 9382 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
W20240502 20:46:53.051220 9382 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55614: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:53.051640 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.051743 9687 raft_consensus.cc:482] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.051803 9687 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Advancing to term 4
W20240502 20:46:53.053109 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51672: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.054288 9687 raft_consensus.cc:504] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.054637 9687 leader_election.cc:288] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 4 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
W20240502 20:46:53.054852 9648 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.054986 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 4 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.055078 9289 raft_consensus.cc:3032] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Stepping down as leader of term 3
I20240502 20:46:53.055120 9289 raft_consensus.cc:729] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.055244 9289 consensus_queue.cc:260] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 3.10, Last appended by leader: 10, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.055531 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 4 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.055629 9421 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.055830 9289 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.056488 9421 raft_consensus.cc:2396] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 4 because replica has last-logged OpId of term: 2 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 9.
I20240502 20:46:53.056674 9289 raft_consensus.cc:2396] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 4 because replica has last-logged OpId of term: 3 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 9.
I20240502 20:46:53.057825 9621 leader_election.cc:302] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:53.057963 11624 raft_consensus.cc:2726] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20240502 20:46:53.058573 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51640: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
W20240502 20:46:53.064447 9648 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40106: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.065369 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.065472 9687 raft_consensus.cc:482] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.065523 9687 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.067286 9687 raft_consensus.cc:504] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.068329 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 4 candidate_status { last_received { term: 3 index: 15 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.068450 9421 raft_consensus.cc:3032] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Stepping down as leader of term 3
I20240502 20:46:53.068491 9421 raft_consensus.cc:729] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.068605 9421 consensus_queue.cc:260] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 3.15, Last appended by leader: 15, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.068774 9421 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Advancing to term 4
W20240502 20:46:53.069507 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59752: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.069607 9421 raft_consensus.cc:2454] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 4.
I20240502 20:46:53.069974 9622 leader_election.cc:302] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:53.070114 11624 raft_consensus.cc:2781] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 20:46:53.070186 11624 raft_consensus.cc:686] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.070307 11624 consensus_queue.cc:237] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 3.15, Last appended by leader: 15, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
W20240502 20:46:53.070354 9382 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55574: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:53.070960 9687 leader_election.cc:288] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 4 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:53.072062 9144 catalog_manager.cc:5484] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 3 to 4, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 4 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
W20240502 20:46:53.072189 9382 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55614: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:53.072923 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 4 candidate_status { last_received { term: 3 index: 15 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.073074 9555 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.074362 9555 raft_consensus.cc:2454] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 4.
I20240502 20:46:53.076598 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.076704 9687 raft_consensus.cc:482] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.076754 9687 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.077512 9687 raft_consensus.cc:504] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.078109 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 4 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.078213 9421 raft_consensus.cc:3032] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Stepping down as leader of term 3
I20240502 20:46:53.078262 9421 raft_consensus.cc:729] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.078388 9421 consensus_queue.cc:260] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 3.7, Last appended by leader: 7, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.078588 9421 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.079402 9421 raft_consensus.cc:2454] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 4.
I20240502 20:46:53.079854 9622 leader_election.cc:302] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:53.080041 11758 raft_consensus.cc:2781] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 20:46:53.080125 11758 raft_consensus.cc:686] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.080246 11758 consensus_queue.cc:237] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 3.7, Last appended by leader: 7, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
W20240502 20:46:53.080773 9382 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:53.081647 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 4 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.081745 9553 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Advancing to term 4
W20240502 20:46:53.081877 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51640: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.082890 9553 raft_consensus.cc:2454] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 4.
W20240502 20:46:53.083017 9648 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40138: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.083247 9687 leader_election.cc:288] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 4 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:53.085291 9144 catalog_manager.cc:5484] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 3 to 4, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 4 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
W20240502 20:46:53.088382 9648 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40106: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
W20240502 20:46:53.093308 9648 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.094331 9555 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.094444 9555 raft_consensus.cc:482] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.094533 9555 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.097258 9555 raft_consensus.cc:504] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
W20240502 20:46:53.097599 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51672: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.098083 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.098212 9421 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.098906 9421 raft_consensus.cc:2454] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 4.
I20240502 20:46:53.099175 9491 leader_election.cc:302] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:53.099345 11577 raft_consensus.cc:2781] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Leader election won for term 4
I20240502 20:46:53.099429 11577 raft_consensus.cc:686] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.099543 11577 consensus_queue.cc:237] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.099614 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.099696 9687 raft_consensus.cc:3032] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 20:46:53.099730 9687 raft_consensus.cc:729] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.099823 9687 consensus_queue.cc:260] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.099933 9687 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.100764 9687 raft_consensus.cc:2396] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20240502 20:46:53.101502 9555 leader_election.cc:288] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 4 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:53.101562 9142 catalog_manager.cc:5484] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 3 to 4, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 4 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
W20240502 20:46:53.103154 9382 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55614: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:53.103940 9555 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.104043 9555 raft_consensus.cc:482] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.104108 9555 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.104943 9555 raft_consensus.cc:504] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.105662 9555 leader_election.cc:288] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.105862 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.105923 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.105962 9289 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.106002 9687 raft_consensus.cc:3032] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 20:46:53.106035 9687 raft_consensus.cc:729] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.106154 9687 consensus_queue.cc:260] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.106259 9687 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.107033 9687 raft_consensus.cc:2396] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20240502 20:46:53.108511 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59752: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.106949 9289 raft_consensus.cc:2396] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20240502 20:46:53.112896 9490 leader_election.cc:302] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448
W20240502 20:46:53.113075 9382 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55574: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:53.113121 11577 raft_consensus.cc:2726] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20240502 20:46:53.113864 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51640: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.116631 9555 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.116734 9555 raft_consensus.cc:482] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.116781 9555 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.117579 9555 raft_consensus.cc:504] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.119990 9555 leader_election.cc:288] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.120451 9284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.120550 9284 raft_consensus.cc:3032] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 LEADER]: Stepping down as leader of term 4
I20240502 20:46:53.120587 9284 raft_consensus.cc:729] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.120703 9284 consensus_queue.cc:260] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.120831 9284 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.121601 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.121701 9421 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.121778 9284 raft_consensus.cc:2396] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 20:46:53.122851 9421 raft_consensus.cc:2396] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 5 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 20:46:53.123136 9491 leader_election.cc:302] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:53.123565 11577 raft_consensus.cc:2726] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20240502 20:46:53.123765 11031 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:53.124143 9555 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.124231 9555 raft_consensus.cc:482] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.124276 9555 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.124315 9667 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 20:46:53.125313 9555 raft_consensus.cc:504] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.125977 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.126073 9687 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.126775 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.126876 9424 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.126945 9687 raft_consensus.cc:2454] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 5.
W20240502 20:46:53.127374 9382 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:53.127738 9492 leader_election.cc:302] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:53.127784 9424 raft_consensus.cc:2454] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 5.
I20240502 20:46:53.127903 11577 raft_consensus.cc:2781] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Leader election won for term 5
I20240502 20:46:53.127977 11577 raft_consensus.cc:686] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.128104 11577 consensus_queue.cc:237] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 3.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
W20240502 20:46:53.128548 9647 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40138: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
W20240502 20:46:53.128953 9648 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40106: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.129086 9555 leader_election.cc:288] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:53.129805 9142 catalog_manager.cc:5484] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 3 to 5, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 5 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:53.130177 9555 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.130268 9555 raft_consensus.cc:482] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.130316 9555 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.132001 9536 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:46:53.137764 11436 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:53.139371 11300 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20240502 20:46:53.140025 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.140055 9404 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:53.141492 11164 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:53.145035 9555 raft_consensus.cc:504] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
W20240502 20:46:53.146306 9382 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55614: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:53.146771 9284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.146871 9284 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.147902 9284 raft_consensus.cc:2454] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 5.
I20240502 20:46:53.148159 9490 leader_election.cc:302] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:53.148304 10554 raft_consensus.cc:2781] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Leader election won for term 5
I20240502 20:46:53.148362 10554 raft_consensus.cc:686] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.148463 10554 consensus_queue.cc:237] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 3.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.150249 9555 leader_election.cc:288] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:53.155968 9136 catalog_manager.cc:5484] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 4 to 5, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 5 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:53.156860 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.156968 9687 raft_consensus.cc:3032] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 20:46:53.157012 9687 raft_consensus.cc:729] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.157130 9687 consensus_queue.cc:260] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.8, Last appended by leader: 8, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.157260 9687 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.157316 9686 raft_consensus.cc:1261] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 9. (index mismatch)
I20240502 20:46:53.158187 9687 raft_consensus.cc:2396] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 5 because replica has last-logged OpId of term: 4 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 7.
I20240502 20:46:53.158840 10554 consensus_queue.cc:1035] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 20:46:53.160013 9424 raft_consensus.cc:1261] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 7. Preceding OpId from leader: term: 5 index: 9. (index mismatch)
I20240502 20:46:53.160430 10554 consensus_queue.cc:1035] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240502 20:46:53.160763 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59752: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
W20240502 20:46:53.161895 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51640: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.162492 9554 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.162590 9554 raft_consensus.cc:482] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.162647 9554 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.163503 9554 raft_consensus.cc:504] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.163821 9554 leader_election.cc:288] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 4 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:53.164181 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 21 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.164268 9686 raft_consensus.cc:3032] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 20:46:53.164306 9686 raft_consensus.cc:729] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.164423 9686 consensus_queue.cc:260] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 3.21, Last appended by leader: 21, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.164546 9686 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.165409 9686 raft_consensus.cc:2454] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 4.
I20240502 20:46:53.165663 9492 leader_election.cc:302] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:53.165827 11577 raft_consensus.cc:2781] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Leader election won for term 4
I20240502 20:46:53.165925 11577 raft_consensus.cc:686] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.166030 11577 consensus_queue.cc:237] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 3.21, Last appended by leader: 21, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.166201 9284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 21 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.167476 9555 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.167601 9555 raft_consensus.cc:482] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.167654 9555 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.168462 9555 raft_consensus.cc:504] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.168639 9136 catalog_manager.cc:5484] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 3 to 4, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 4 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:53.168777 9555 leader_election.cc:288] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.169889 9284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.170010 9284 raft_consensus.cc:3032] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 LEADER]: Stepping down as leader of term 4
I20240502 20:46:53.170059 9284 raft_consensus.cc:729] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.170140 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.170189 9284 consensus_queue.cc:260] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.170295 9424 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.170341 9284 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.171272 9284 raft_consensus.cc:2396] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20240502 20:46:53.174306 9554 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3315525f1849462fbf49a285f424ea51"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.174413 9554 raft_consensus.cc:482] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.174495 9554 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.175338 9554 raft_consensus.cc:504] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.175653 9554 leader_election.cc:288] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.176407 9554 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.176527 9554 raft_consensus.cc:482] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.176584 9554 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.178177 9555 raft_consensus.cc:1226] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Rejecting Update request from peer 4b3cb6ad00954dc5a1270dc392870b4f for earlier term 4. Current term is 5. Ops: []
I20240502 20:46:53.178440 9554 raft_consensus.cc:504] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.178778 9554 leader_election.cc:288] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.179061 11739 consensus_queue.cc:1046] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: INVALID_TERM, Last received: 4.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.000s
I20240502 20:46:53.179617 11725 raft_consensus.cc:3032] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 LEADER]: Stepping down as leader of term 4
I20240502 20:46:53.179677 11725 raft_consensus.cc:729] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.179800 11725 consensus_queue.cc:260] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 4.20, Last appended by leader: 20, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.179950 11725 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Advancing to term 5
W20240502 20:46:53.179992 9382 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:53.180459 9284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 9 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.180547 9284 raft_consensus.cc:3032] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 LEADER]: Stepping down as leader of term 4
I20240502 20:46:53.180581 9284 raft_consensus.cc:729] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.180680 9284 consensus_queue.cc:260] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 4.10, Last appended by leader: 10, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.180797 9284 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.181593 9284 raft_consensus.cc:2396] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 5 because replica has last-logged OpId of term: 4 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 9.
I20240502 20:46:53.181815 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 9 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.181905 9423 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.181968 9555 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.182057 9555 raft_consensus.cc:482] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.182098 9555 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.182847 9423 raft_consensus.cc:2396] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 5 because replica has last-logged OpId of term: 3 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 9.
I20240502 20:46:53.182842 9555 raft_consensus.cc:504] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.183145 9269 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 20:46:53.183360 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 4 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.183447 9686 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.183740 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3315525f1849462fbf49a285f424ea51" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 20 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.183831 9421 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.184228 9686 raft_consensus.cc:2454] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 4.
I20240502 20:46:53.184484 9492 leader_election.cc:302] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:53.184612 9421 raft_consensus.cc:2454] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 5.
I20240502 20:46:53.184619 11577 raft_consensus.cc:2781] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Leader election won for term 4
I20240502 20:46:53.184687 11577 raft_consensus.cc:686] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.184784 11577 consensus_queue.cc:237] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 8, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.185600 9555 leader_election.cc:288] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 4 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:53.185640 9491 leader_election.cc:302] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:53.185879 9491 leader_election.cc:302] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:53.186165 11578 raft_consensus.cc:2726] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20240502 20:46:53.186251 11578 raft_consensus.cc:2781] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Leader election won for term 5
I20240502 20:46:53.186322 11578 raft_consensus.cc:686] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.186426 11578 consensus_queue.cc:237] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 4.20, Last appended by leader: 20, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
W20240502 20:46:53.186674 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59778: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.186743 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3315525f1849462fbf49a285f424ea51" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 20 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.187788 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 4 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.187872 9421 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.187844 9136 catalog_manager.cc:5484] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 2 to 4, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 4 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:53.188627 9421 raft_consensus.cc:2454] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 4.
I20240502 20:46:53.190294 9424 raft_consensus.cc:2396] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20240502 20:46:53.190627 9491 leader_election.cc:302] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:53.190811 10554 raft_consensus.cc:2726] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20240502 20:46:53.192471 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40106: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
W20240502 20:46:53.193527 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40118: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.193871 9136 catalog_manager.cc:5484] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 4 to 5, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 5 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:53.193975 9686 raft_consensus.cc:1261] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 8. Preceding OpId from leader: term: 5 index: 9. (index mismatch)
I20240502 20:46:53.194051 9284 raft_consensus.cc:1261] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 7. Preceding OpId from leader: term: 5 index: 9. (index mismatch)
I20240502 20:46:53.194341 10554 consensus_queue.cc:1035] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 20:46:53.194437 10554 consensus_queue.cc:1225] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Peer 64136bd82a30486e9d56fc28da4bd448 log is divergent from this leader: its last log entry 4.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20240502 20:46:53.194904 11577 consensus_queue.cc:1035] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 20:46:53.195376 9686 pending_rounds.cc:76] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448: Aborting all ops after (but not including) 7
I20240502 20:46:53.195470 9686 pending_rounds.cc:97] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448: Aborting uncommitted NO_OP operation due to leader change: 4.8
I20240502 20:46:53.195541 9686 raft_consensus.cc:2866] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 20:46:53.196156 9554 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "edebc185b24441da9504c78394378f4d"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.196592 9554 raft_consensus.cc:482] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.196640 9554 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Advancing to term 5
W20240502 20:46:53.198503 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.199924 9554 raft_consensus.cc:504] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.200486 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edebc185b24441da9504c78394378f4d" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.200584 9424 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.201349 9424 raft_consensus.cc:2454] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 5.
I20240502 20:46:53.204164 9491 leader_election.cc:302] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:53.204300 10554 raft_consensus.cc:2781] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Leader election won for term 5
I20240502 20:46:53.204356 9554 leader_election.cc:288] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:53.204370 10554 raft_consensus.cc:686] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.204481 10554 consensus_queue.cc:237] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 3.12, Last appended by leader: 12, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.204963 9289 raft_consensus.cc:2454] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 5.
I20240502 20:46:53.206243 9136 catalog_manager.cc:5484] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 4 to 5, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 5 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:53.207433 9687 raft_consensus.cc:1261] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 21. Preceding OpId from leader: term: 4 index: 23. (index mismatch)
I20240502 20:46:53.208909 11532 consensus_queue.cc:1035] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20240502 20:46:53.210112 11504 heartbeater.cc:503] Master 127.22.45.126:41377 was elected leader, sending a full tablet report...
I20240502 20:46:53.213410 11536 mvcc.cc:205] Tried to move back new op lower bound from 7023340802864394240 to 7023340802728247296. Current Snapshot: MvccSnapshot[applied={T|T < 7023340802864394240}]
W20240502 20:46:53.215741 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59728: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.217052 9421 raft_consensus.cc:1261] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 4 index: 10. (index mismatch)
W20240502 20:46:53.217162 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59752: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.217800 9688 raft_consensus.cc:1261] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 4 index: 10. (index mismatch)
I20240502 20:46:53.219357 11577 consensus_queue.cc:1035] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 20:46:53.219552 9554 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.219700 9554 raft_consensus.cc:482] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.219877 9554 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.220342 9286 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.220795 9554 raft_consensus.cc:504] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.221334 9286 raft_consensus.cc:1261] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 21. Preceding OpId from leader: term: 4 index: 23. (index mismatch)
I20240502 20:46:53.222625 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.222715 9687 raft_consensus.cc:3032] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 20:46:53.222746 9687 raft_consensus.cc:729] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.222867 9687 consensus_queue.cc:260] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 3.12, Last appended by leader: 12, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.222973 9687 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.223826 9687 raft_consensus.cc:2396] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 4 because replica has last-logged OpId of term: 3 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 11.
I20240502 20:46:53.223845 11577 consensus_queue.cc:1035] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 20:46:53.224490 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edebc185b24441da9504c78394378f4d" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.224584 9686 raft_consensus.cc:3032] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 20:46:53.224618 9686 raft_consensus.cc:729] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.224736 9686 consensus_queue.cc:260] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 4.13, Last appended by leader: 13, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.224862 9686 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.225713 9686 raft_consensus.cc:2396] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 5 because replica has last-logged OpId of term: 4 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 12.
W20240502 20:46:53.227370 9382 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55522: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
W20240502 20:46:53.234115 9382 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55614: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:53.236456 9424 raft_consensus.cc:1261] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 20. Preceding OpId from leader: term: 5 index: 22. (index mismatch)
I20240502 20:46:53.236801 10554 consensus_queue.cc:1035] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
W20240502 20:46:53.238022 9382 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
W20240502 20:46:53.238023 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59786: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.238582 11577 consensus_queue.cc:1035] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.000s
I20240502 20:46:53.239643 9284 raft_consensus.cc:1261] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 20. Preceding OpId from leader: term: 5 index: 22. (index mismatch)
I20240502 20:46:53.240193 9554 leader_election.cc:288] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 4 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.240943 9554 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.241043 9554 raft_consensus.cc:482] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.241093 9554 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.242092 9554 raft_consensus.cc:504] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.242589 9554 leader_election.cc:288] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 4 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:53.242983 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 4 candidate_status { last_received { term: 2 index: 10 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.243038 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 4 candidate_status { last_received { term: 2 index: 10 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.243079 9686 raft_consensus.cc:3032] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 20:46:53.243114 9686 raft_consensus.cc:729] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.243132 9421 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.243229 9686 consensus_queue.cc:260] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 3.11, Last appended by leader: 11, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.243347 9686 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.244143 9421 raft_consensus.cc:2454] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 4.
I20240502 20:46:53.244463 9491 leader_election.cc:302] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:53.244614 10554 raft_consensus.cc:2781] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Leader election won for term 4
I20240502 20:46:53.244688 10554 raft_consensus.cc:686] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.244802 10554 consensus_queue.cc:237] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 2.10, Last appended by leader: 10, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
W20240502 20:46:53.245980 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51640: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.246515 9686 raft_consensus.cc:2396] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 4 because replica has last-logged OpId of term: 3 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 10.
I20240502 20:46:53.247437 9144 catalog_manager.cc:5484] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 3 to 4, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 4 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:53.250358 11578 consensus_queue.cc:1035] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20240502 20:46:53.250579 9555 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.250684 9555 raft_consensus.cc:482] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.250746 9555 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.251261 9284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.251408 9284 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.251633 9555 raft_consensus.cc:504] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.251935 9555 leader_election.cc:288] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:53.254333 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 16 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.255383 9555 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "20698b2531674eefb9df3125a86653ba"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.255483 9555 raft_consensus.cc:482] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.255532 9555 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.256300 9555 raft_consensus.cc:504] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.256603 9555 leader_election.cc:288] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.257035 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20698b2531674eefb9df3125a86653ba" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 15 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.257133 9287 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.257323 9284 raft_consensus.cc:2396] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 4 because replica has last-logged OpId of term: 2 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 11.
W20240502 20:46:53.257563 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.257846 9287 raft_consensus.cc:2454] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 5.
I20240502 20:46:53.258152 9555 raft_consensus.cc:1226] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Rejecting Update request from peer 64136bd82a30486e9d56fc28da4bd448 for earlier term 4. Current term is 5. Ops: []
I20240502 20:46:53.258555 11758 consensus_queue.cc:1046] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: INVALID_TERM, Last received: 4.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20240502 20:46:53.258700 11758 raft_consensus.cc:3032] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Stepping down as leader of term 4
W20240502 20:46:53.258713 9382 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55614: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:53.258745 11758 raft_consensus.cc:729] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.258862 11758 consensus_queue.cc:260] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 4.16, Last appended by leader: 16, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.258993 11758 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.261014 9490 leader_election.cc:302] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:53.261529 9490 leader_election.cc:302] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:53.262042 11577 raft_consensus.cc:2726] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 20:46:53.262121 11577 raft_consensus.cc:2781] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Leader election won for term 5
I20240502 20:46:53.262161 11577 raft_consensus.cc:686] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.262207 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20698b2531674eefb9df3125a86653ba" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 15 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.262249 11577 consensus_queue.cc:237] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 3.15, Last appended by leader: 15, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.262316 9423 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.262542 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 16 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.263222 9423 raft_consensus.cc:2454] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 5.
I20240502 20:46:53.264014 9554 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.264106 9554 raft_consensus.cc:482] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.264155 9554 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Advancing to term 5
W20240502 20:46:53.265182 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59786: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.265141 9144 catalog_manager.cc:5484] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 3 to 5, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 5 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:53.265792 9554 raft_consensus.cc:504] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.266067 9554 leader_election.cc:288] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:53.267041 9689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.267132 9689 raft_consensus.cc:3032] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 20:46:53.267169 9689 raft_consensus.cc:729] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.267306 9689 consensus_queue.cc:260] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.267431 9689 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.267639 9686 raft_consensus.cc:2454] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 5.
I20240502 20:46:53.268280 9689 raft_consensus.cc:2396] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20240502 20:46:53.269667 9554 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.269964 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.270118 9424 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.270206 9554 raft_consensus.cc:482] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.270270 9554 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.270540 9492 leader_election.cc:302] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters: 4b3cb6ad00954dc5a1270dc392870b4f
I20240502 20:46:53.270702 11577 raft_consensus.cc:2781] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Leader election won for term 5
I20240502 20:46:53.270787 11577 raft_consensus.cc:686] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.270898 11577 consensus_queue.cc:237] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 4.16, Last appended by leader: 16, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.271044 9554 raft_consensus.cc:504] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.271294 9554 leader_election.cc:288] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:53.271857 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.271952 9287 raft_consensus.cc:3032] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 LEADER]: Stepping down as leader of term 4
I20240502 20:46:53.271987 9287 raft_consensus.cc:729] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.272121 9287 consensus_queue.cc:260] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 4.13, Last appended by leader: 13, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.272249 9287 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.273265 9287 raft_consensus.cc:2396] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 5 because replica has last-logged OpId of term: 4 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 12.
I20240502 20:46:53.275014 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.275115 9423 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.275796 9555 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.275893 9555 raft_consensus.cc:482] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.275944 9555 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.276077 9423 raft_consensus.cc:2454] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 5.
I20240502 20:46:53.276412 9491 leader_election.cc:302] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:53.276564 9424 raft_consensus.cc:2454] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 5.
I20240502 20:46:53.276692 9555 raft_consensus.cc:504] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.276983 9555 leader_election.cc:288] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:53.277103 11578 raft_consensus.cc:2781] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Leader election won for term 5
I20240502 20:46:53.277161 11578 raft_consensus.cc:686] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.277266 11578 consensus_queue.cc:237] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 3.12, Last appended by leader: 12, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.277410 9491 leader_election.cc:302] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 64136bd82a30486e9d56fc28da4bd448
W20240502 20:46:53.278270 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51640: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.278616 11577 raft_consensus.cc:2781] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Leader election won for term 5
I20240502 20:46:53.278657 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 2 index: 15 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.278705 11577 raft_consensus.cc:686] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.278748 9687 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.278841 11577 consensus_queue.cc:237] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.279601 9687 raft_consensus.cc:2454] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 5.
I20240502 20:46:53.279815 9492 leader_election.cc:302] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:53.279840 9554 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.279917 9554 raft_consensus.cc:482] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.279961 9554 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Advancing to term 5
W20240502 20:46:53.280481 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59752: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.280673 9554 raft_consensus.cc:504] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.281129 9554 leader_election.cc:288] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:53.281155 11578 raft_consensus.cc:2781] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Leader election won for term 5
I20240502 20:46:53.281225 11578 raft_consensus.cc:686] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.281314 11578 consensus_queue.cc:237] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.15, Last appended by leader: 15, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.281607 9554 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.281692 9554 raft_consensus.cc:482] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.281734 9554 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.282413 9554 raft_consensus.cc:504] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.282718 9554 leader_election.cc:288] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:53.283233 9555 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.283327 9555 raft_consensus.cc:482] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.283375 9555 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.284085 9555 raft_consensus.cc:504] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.284255 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 15 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.284343 9421 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.284387 9555 leader_election.cc:288] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.284840 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.284932 9423 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.285082 9555 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.285171 9555 raft_consensus.cc:482] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.285218 9555 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.285204 9421 raft_consensus.cc:2454] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 5.
I20240502 20:46:53.285712 9423 raft_consensus.cc:2396] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 5 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20240502 20:46:53.286007 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.286100 9424 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.286108 9555 raft_consensus.cc:504] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.286387 9555 leader_election.cc:288] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 4 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:53.286970 9555 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.287065 9555 raft_consensus.cc:482] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.287153 9555 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.287807 9284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.287856 9555 raft_consensus.cc:504] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.287922 9284 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.288148 9555 leader_election.cc:288] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 4 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.288642 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.288686 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 15 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.288741 9686 raft_consensus.cc:3032] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 20:46:53.288786 9686 raft_consensus.cc:729] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.288924 9284 raft_consensus.cc:2396] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 4 because replica has last-logged OpId of term: 2 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 15.
I20240502 20:46:53.288901 9686 consensus_queue.cc:260] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 3.16, Last appended by leader: 16, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.287696 9424 raft_consensus.cc:2454] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 5.
I20240502 20:46:53.289026 9686 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.289547 9554 raft_consensus.cc:1226] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Rejecting Update request from peer 4b3cb6ad00954dc5a1270dc392870b4f for earlier term 3. Current term is 4. Ops: [3.13-3.13]
I20240502 20:46:53.289862 9686 raft_consensus.cc:2396] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 4 because replica has last-logged OpId of term: 3 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 15.
I20240502 20:46:53.289892 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.289984 9289 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.288642 9689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.290329 9689 raft_consensus.cc:3032] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 20:46:53.290366 9689 raft_consensus.cc:729] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.290351 11725 consensus_queue.cc:1046] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: INVALID_TERM, Last received: 3.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 20:46:53.290541 11725 raft_consensus.cc:3032] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Stepping down as leader of term 3
I20240502 20:46:53.290524 9689 consensus_queue.cc:260] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.8, Last appended by leader: 8, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.290585 11725 raft_consensus.cc:729] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.290643 9689 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.290712 11725 consensus_queue.cc:260] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 3.13, Last appended by leader: 13, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.290844 11725 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.290942 9289 raft_consensus.cc:2454] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 5.
I20240502 20:46:53.291256 9689 raft_consensus.cc:2396] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 5 because replica has last-logged OpId of term: 4 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 7.
W20240502 20:46:53.291283 11558 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59778: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.003s ago.
I20240502 20:46:53.288745 9687 raft_consensus.cc:3032] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 20:46:53.291596 9687 raft_consensus.cc:729] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.291711 9687 consensus_queue.cc:260] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 4.16, Last appended by leader: 16, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.291831 9687 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.292132 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.292269 9424 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.292682 9687 raft_consensus.cc:2396] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 5 because replica has last-logged OpId of term: 4 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 15.
I20240502 20:46:53.293393 9424 raft_consensus.cc:2396] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 4 because replica has last-logged OpId of term: 3 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 12.
W20240502 20:46:53.294072 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55614: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:53.294294 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 5 candidate_status { last_received { term: 2 index: 15 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.294384 9287 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.294575 9490 leader_election.cc:302] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:53.294708 11578 raft_consensus.cc:2726] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 20:46:53.294878 9490 leader_election.cc:302] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:53.294999 11578 raft_consensus.cc:2781] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Leader election won for term 5
I20240502 20:46:53.295027 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.295056 11578 raft_consensus.cc:686] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.295137 9289 raft_consensus.cc:2396] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 4 because replica has last-logged OpId of term: 3 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 12.
I20240502 20:46:53.295156 11578 consensus_queue.cc:237] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.295495 9424 raft_consensus.cc:1226] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Rejecting Update request from peer 4b3cb6ad00954dc5a1270dc392870b4f for earlier term 3. Current term is 4. Ops: []
I20240502 20:46:53.295662 9423 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:53.295768 9423 raft_consensus.cc:482] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.295828 9423 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.296247 9287 raft_consensus.cc:2396] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 5 because replica has last-logged OpId of term: 3 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 15.
I20240502 20:46:53.296448 9491 leader_election.cc:302] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:53.296612 11578 raft_consensus.cc:2781] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Leader election won for term 5
I20240502 20:46:53.296694 11578 raft_consensus.cc:686] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.296820 9491 leader_election.cc:302] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:53.297004 9423 raft_consensus.cc:504] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.297580 9423 leader_election.cc:288] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 6 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.297756 10554 raft_consensus.cc:2781] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Leader election won for term 5
I20240502 20:46:53.297853 10554 raft_consensus.cc:686] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
W20240502 20:46:53.297968 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51672: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.297959 10554 consensus_queue.cc:237] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 3.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.298164 9424 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:53.296802 11578 consensus_queue.cc:237] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 3.15, Last appended by leader: 15, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.298358 9424 raft_consensus.cc:482] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.298660 9424 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Advancing to term 4
W20240502 20:46:53.298982 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40118: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.299216 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.299476 9424 raft_consensus.cc:504] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.299546 9491 leader_election.cc:302] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:53.299690 11578 raft_consensus.cc:2726] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 20:46:53.300117 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 4 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.300220 9687 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.300290 9136 catalog_manager.cc:5484] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 4 to 5, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 5 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:53.300985 9687 raft_consensus.cc:2454] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 4.
I20240502 20:46:53.301277 9357 leader_election.cc:302] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:53.301446 11710 raft_consensus.cc:2781] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Leader election won for term 4
I20240502 20:46:53.301518 11710 raft_consensus.cc:686] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.301668 9287 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.301632 11710 consensus_queue.cc:237] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.302609 9287 raft_consensus.cc:2396] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20240502 20:46:53.302874 9424 leader_election.cc:288] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 4 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.303059 9554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.303160 9554 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Advancing to term 6
W20240502 20:46:53.303572 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:53.304181 9554 raft_consensus.cc:2454] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 6.
I20240502 20:46:53.304164 9136 catalog_manager.cc:5484] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 2 to 4, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 4 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
W20240502 20:46:53.304312 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55574: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:53.304461 9357 leader_election.cc:302] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 4b3cb6ad00954dc5a1270dc392870b4f
I20240502 20:46:53.304658 11710 raft_consensus.cc:2781] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Leader election won for term 6
I20240502 20:46:53.304729 11710 raft_consensus.cc:686] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 6 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
W20240502 20:46:53.304715 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59728: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.304852 11710 consensus_queue.cc:237] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.305322 9424 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:53.305418 9424 raft_consensus.cc:482] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.305464 9424 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.305694 9144 catalog_manager.cc:5484] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 4 to 5, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 5 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:53.305899 9144 catalog_manager.cc:5484] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 3 to 5, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 5 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:53.306075 9144 catalog_manager.cc:5484] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 4 to 5, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 5 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:53.306236 9144 catalog_manager.cc:5484] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 3 to 5, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 5 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:53.306298 9424 raft_consensus.cc:504] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.306386 9144 catalog_manager.cc:5484] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 4 to 5, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 5 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:53.306664 9144 catalog_manager.cc:5484] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 4 to 5, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 5 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:53.306857 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 6 candidate_status { last_received { term: 5 index: 14 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.306946 9687 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Advancing to term 6
W20240502 20:46:53.307206 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59786: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.307598 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 4 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.307718 9287 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.307749 9687 raft_consensus.cc:2454] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 6.
I20240502 20:46:53.308068 9357 leader_election.cc:302] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:53.308229 11712 raft_consensus.cc:2781] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Leader election won for term 6
I20240502 20:46:53.308321 11712 raft_consensus.cc:686] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.308436 11712 consensus_queue.cc:237] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 5.14, Last appended by leader: 14, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.308576 9287 raft_consensus.cc:2396] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 5.
I20240502 20:46:53.307220 9136 catalog_manager.cc:5484] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 4 to 6, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 6 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:53.309299 9287 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.310366 9687 raft_consensus.cc:1261] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 16. Preceding OpId from leader: term: 5 index: 18. (index mismatch)
I20240502 20:46:53.310827 9424 leader_election.cc:288] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 6 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:53.311112 9554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 6 candidate_status { last_received { term: 5 index: 14 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.311182 9287 raft_consensus.cc:1261] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 16. Preceding OpId from leader: term: 5 index: 18. (index mismatch)
I20240502 20:46:53.311209 9554 raft_consensus.cc:3032] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Stepping down as leader of term 5
I20240502 20:46:53.311249 9554 raft_consensus.cc:729] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.311367 9554 consensus_queue.cc:260] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 5.14, Last appended by leader: 14, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.311496 9423 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "edebc185b24441da9504c78394378f4d"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:53.311584 9423 raft_consensus.cc:482] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.311548 11577 consensus_queue.cc:1035] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240502 20:46:53.311632 9423 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.313045 9144 catalog_manager.cc:5484] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 5 to 6, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 6 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:53.313968 11778 consensus_queue.cc:1035] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240502 20:46:53.314234 9423 raft_consensus.cc:504] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.314566 9554 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Advancing to term 6
W20240502 20:46:53.314699 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59778: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.314797 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edebc185b24441da9504c78394378f4d" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 6 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.314878 9687 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.315361 9554 raft_consensus.cc:2454] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 6.
I20240502 20:46:53.315663 9687 raft_consensus.cc:2396] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 6 because replica has last-logged OpId of term: 4 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 12.
I20240502 20:46:53.316361 9423 leader_election.cc:288] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 6 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:53.318636 9421 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:53.318747 9421 raft_consensus.cc:482] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.318811 9421 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.319510 9554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edebc185b24441da9504c78394378f4d" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 6 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.319608 9554 raft_consensus.cc:3032] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Stepping down as leader of term 5
I20240502 20:46:53.319658 9554 raft_consensus.cc:729] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.319670 9421 raft_consensus.cc:504] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.319977 9421 leader_election.cc:288] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 6 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:53.319818 9284 raft_consensus.cc:1261] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20240502 20:46:53.319973 9554 consensus_queue.cc:260] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 5.13, Last appended by leader: 13, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.320103 9554 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.320365 9421 raft_consensus.cc:1261] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20240502 20:46:53.320524 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51672: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.321110 9554 raft_consensus.cc:2396] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 6 because replica has last-logged OpId of term: 5 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 12.
I20240502 20:46:53.321061 11577 consensus_queue.cc:1035] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:53.321173 11577 consensus_queue.cc:1225] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Peer 88a19090795d47c2a1ea1dd979fb62bc log is divergent from this leader: its last log entry 3.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20240502 20:46:53.321432 11577 consensus_queue.cc:1035] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 20:46:53.322386 9421 pending_rounds.cc:76] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc: Aborting all ops after (but not including) 3
I20240502 20:46:53.322432 9421 pending_rounds.cc:97] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc: Aborting uncommitted NO_OP operation due to leader change: 3.4
I20240502 20:46:53.322502 9421 raft_consensus.cc:2866] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 20:46:53.323799 9357 leader_election.cc:302] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:53.323972 11710 raft_consensus.cc:2726] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20240502 20:46:53.324393 9420 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3315525f1849462fbf49a285f424ea51"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:53.324512 9420 raft_consensus.cc:482] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.324575 9420 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.325495 9420 raft_consensus.cc:504] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.325767 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.325896 9420 leader_election.cc:288] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 6 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.325896 9287 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.326535 9554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.326633 9554 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Advancing to term 6
W20240502 20:46:53.327256 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.327931 9284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3315525f1849462fbf49a285f424ea51" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 6 candidate_status { last_received { term: 5 index: 24 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.328090 9284 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.328280 9554 raft_consensus.cc:2454] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 6.
I20240502 20:46:53.328330 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3315525f1849462fbf49a285f424ea51" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 6 candidate_status { last_received { term: 5 index: 24 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.328442 9555 raft_consensus.cc:3032] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Stepping down as leader of term 5
I20240502 20:46:53.328495 9555 raft_consensus.cc:729] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.328615 9555 consensus_queue.cc:260] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 5.24, Last appended by leader: 24, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.328784 9555 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.329138 9284 raft_consensus.cc:2454] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 6.
I20240502 20:46:53.329411 9355 leader_election.cc:302] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:53.329571 11710 raft_consensus.cc:2781] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Leader election won for term 6
I20240502 20:46:53.329583 9287 raft_consensus.cc:2396] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20240502 20:46:53.329624 9555 raft_consensus.cc:2454] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 6.
I20240502 20:46:53.329639 11710 raft_consensus.cc:686] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.329741 11710 consensus_queue.cc:237] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 5.24, Last appended by leader: 24, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
W20240502 20:46:53.330103 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55574: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:53.331068 9357 leader_election.cc:302] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 4b3cb6ad00954dc5a1270dc392870b4f
I20240502 20:46:53.331427 11712 raft_consensus.cc:2781] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Leader election won for term 6
I20240502 20:46:53.331518 11712 raft_consensus.cc:686] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 6 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.331633 11712 consensus_queue.cc:237] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.331732 9144 catalog_manager.cc:5484] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 5 to 6, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 6 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:53.333581 11707 mvcc.cc:205] Tried to move back new op lower bound from 7023340803345727488 to 7023340803257135104. Current Snapshot: MvccSnapshot[applied={T|T < 7023340803345727488}]
I20240502 20:46:53.334136 9424 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:53.334233 9424 raft_consensus.cc:482] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.334292 9424 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.334998 9424 raft_consensus.cc:504] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.335260 9424 leader_election.cc:288] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 6 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:53.335669 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.335790 9687 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Advancing to term 6
W20240502 20:46:53.336278 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51610: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.336642 9687 raft_consensus.cc:2396] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20240502 20:46:53.336637 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.336719 9555 raft_consensus.cc:3032] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Stepping down as leader of term 5
I20240502 20:46:53.336752 9555 raft_consensus.cc:729] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.336895 9555 consensus_queue.cc:260] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.337028 9555 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.337363 9424 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:53.337461 9424 raft_consensus.cc:482] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.337509 9424 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.337862 9555 raft_consensus.cc:2396] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20240502 20:46:53.339388 9357 leader_election.cc:302] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:53.339731 11712 raft_consensus.cc:2726] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20240502 20:46:53.340025 9424 raft_consensus.cc:504] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.340674 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 6 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.340768 9555 raft_consensus.cc:3032] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Stepping down as leader of term 5
I20240502 20:46:53.340817 9555 raft_consensus.cc:729] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.341064 9424 leader_election.cc:288] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 6 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.340931 9555 consensus_queue.cc:260] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 5.13, Last appended by leader: 13, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.341535 9555 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.341744 9424 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:53.341845 9424 raft_consensus.cc:482] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.342356 9424 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.342682 9555 raft_consensus.cc:2396] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 6 because replica has last-logged OpId of term: 5 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 12.
I20240502 20:46:53.343133 9424 raft_consensus.cc:504] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.343169 9554 raft_consensus.cc:1261] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 5 index: 24. Preceding OpId from leader: term: 6 index: 26. (index mismatch)
I20240502 20:46:53.343448 9424 leader_election.cc:288] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 5 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
W20240502 20:46:53.344117 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59778: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.344177 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 5 candidate_status { last_received { term: 4 index: 11 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.344267 9687 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.344473 9139 catalog_manager.cc:5484] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 4 to 6, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 6 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:53.345084 9687 raft_consensus.cc:2454] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 5.
I20240502 20:46:53.345759 9357 leader_election.cc:302] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:53.345997 11712 raft_consensus.cc:2781] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Leader election won for term 5
I20240502 20:46:53.346073 11712 raft_consensus.cc:686] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.346184 11712 consensus_queue.cc:237] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 4.11, Last appended by leader: 11, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.346746 9289 raft_consensus.cc:1261] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 5 index: 24. Preceding OpId from leader: term: 6 index: 26. (index mismatch)
I20240502 20:46:53.347038 9420 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fe22224142984d69a2def286584b9aaa"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:53.347128 9420 raft_consensus.cc:482] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.347175 9420 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.347182 11712 consensus_queue.cc:1035] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.000s
I20240502 20:46:53.347741 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 6 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.347832 9289 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.347959 9420 raft_consensus.cc:504] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.348502 9420 leader_election.cc:288] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 5 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:53.348742 9289 raft_consensus.cc:2396] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 6 because replica has last-logged OpId of term: 4 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 12.
I20240502 20:46:53.349215 9420 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:53.349260 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe22224142984d69a2def286584b9aaa" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 5 candidate_status { last_received { term: 2 index: 19 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.349342 9687 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.349462 9420 raft_consensus.cc:482] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.349522 9420 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.349937 9144 catalog_manager.cc:5484] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 4 to 5, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 5 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:53.350100 9687 raft_consensus.cc:2454] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 5.
I20240502 20:46:53.350348 9357 leader_election.cc:302] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:53.350534 11712 raft_consensus.cc:2781] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Leader election won for term 5
I20240502 20:46:53.350612 11712 raft_consensus.cc:686] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 5 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.350721 11712 consensus_queue.cc:237] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 2.19, Last appended by leader: 19, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.351050 9355 leader_election.cc:302] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f
I20240502 20:46:53.351197 11712 raft_consensus.cc:2726] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20240502 20:46:53.351408 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 5 candidate_status { last_received { term: 4 index: 11 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.351503 9555 raft_consensus.cc:3032] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 LEADER]: Stepping down as leader of term 4
I20240502 20:46:53.351537 9555 raft_consensus.cc:729] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.351683 9555 consensus_queue.cc:260] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 4.11, Last appended by leader: 11, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.351852 9555 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.351908 9420 raft_consensus.cc:504] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.352317 9420 leader_election.cc:288] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 5 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:53.352535 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 5 candidate_status { last_received { term: 2 index: 10 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.352581 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe22224142984d69a2def286584b9aaa" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 5 candidate_status { last_received { term: 2 index: 19 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.352636 9553 raft_consensus.cc:3032] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 LEADER]: Stepping down as leader of term 4
I20240502 20:46:53.352674 9289 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.352694 9553 raft_consensus.cc:729] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.352813 9553 consensus_queue.cc:260] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 4.11, Last appended by leader: 11, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.352945 9553 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.353758 9289 raft_consensus.cc:2396] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 5 because replica has last-logged OpId of term: 3 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 19.
I20240502 20:46:53.353785 9553 raft_consensus.cc:2396] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 5 because replica has last-logged OpId of term: 4 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 10.
I20240502 20:46:53.354101 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 5 candidate_status { last_received { term: 2 index: 10 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.354202 9687 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.355017 9687 raft_consensus.cc:2396] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 5 because replica has last-logged OpId of term: 3 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 10.
W20240502 20:46:53.355142 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59752: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.355324 9357 leader_election.cc:302] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:53.355437 9555 raft_consensus.cc:2454] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 5.
I20240502 20:46:53.355509 11710 raft_consensus.cc:2726] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20240502 20:46:53.355733 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51672: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.355788 9420 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:53.355892 9420 raft_consensus.cc:482] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.355949 9420 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.356328 9139 catalog_manager.cc:5484] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 3 to 5, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 5 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:53.356535 9420 raft_consensus.cc:504] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.357088 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 5 candidate_status { last_received { term: 4 index: 16 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.357223 9687 raft_consensus.cc:3032] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 20:46:53.357304 9687 raft_consensus.cc:729] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.357975 9687 consensus_queue.cc:260] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 4.16, Last appended by leader: 16, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.358058 9420 leader_election.cc:288] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 5 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.358218 9687 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.359059 11710 consensus_queue.cc:1035] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.000s
I20240502 20:46:53.359092 9687 raft_consensus.cc:2454] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 5.
I20240502 20:46:53.359423 9357 leader_election.cc:302] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:53.359556 11710 raft_consensus.cc:2781] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Leader election won for term 5
I20240502 20:46:53.359616 11710 raft_consensus.cc:686] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.359710 11710 consensus_queue.cc:237] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 4.16, Last appended by leader: 16, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.360942 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 5 candidate_status { last_received { term: 4 index: 16 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.361088 9287 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.361546 9144 catalog_manager.cc:5484] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 4 to 5, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 5 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:53.362605 9287 raft_consensus.cc:2454] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 5.
W20240502 20:46:53.368819 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55574: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
W20240502 20:46:53.374331 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51640: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.378543 9420 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:53.378645 9420 raft_consensus.cc:482] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.378701 9420 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Advancing to term 6
W20240502 20:46:53.379354 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40106: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.379423 9420 raft_consensus.cc:504] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.380017 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 6 candidate_status { last_received { term: 3 index: 10 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.380106 9555 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.380906 9555 raft_consensus.cc:2454] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 6.
I20240502 20:46:53.381139 9357 leader_election.cc:302] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:53.381291 11710 raft_consensus.cc:2781] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Leader election won for term 6
I20240502 20:46:53.381366 11710 raft_consensus.cc:686] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 6 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.381474 11710 consensus_queue.cc:237] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 3.10, Last appended by leader: 10, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.381853 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 6 candidate_status { last_received { term: 3 index: 10 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.381953 9289 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.382164 9420 leader_election.cc:288] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 6 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
W20240502 20:46:53.382381 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:53.383327 9289 raft_consensus.cc:2396] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 6 because replica has last-logged OpId of term: 4 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 10.
I20240502 20:46:53.383877 9144 catalog_manager.cc:5484] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 4 to 6, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 6 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:53.386251 9687 raft_consensus.cc:1261] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 4 index: 11. Preceding OpId from leader: term: 5 index: 13. (index mismatch)
W20240502 20:46:53.386382 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59778: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.386682 11710 consensus_queue.cc:1035] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 20:46:53.386785 9420 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:53.386891 9420 raft_consensus.cc:482] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.386946 9420 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.387256 9555 raft_consensus.cc:1261] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 4 index: 11. Preceding OpId from leader: term: 5 index: 13. (index mismatch)
I20240502 20:46:53.387755 9420 raft_consensus.cc:504] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.388036 9420 leader_election.cc:288] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 5 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:53.388749 9689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.388742 11710 consensus_queue.cc:1035] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 20:46:53.388856 9689 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.389685 9689 raft_consensus.cc:2396] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20240502 20:46:53.390560 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.390888 9555 raft_consensus.cc:3032] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 LEADER]: Stepping down as leader of term 4
I20240502 20:46:53.390952 9555 raft_consensus.cc:729] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.391099 9555 consensus_queue.cc:260] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.391237 9555 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.392083 9555 raft_consensus.cc:2396] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20240502 20:46:53.394131 9357 leader_election.cc:302] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:53.394259 11712 raft_consensus.cc:2726] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20240502 20:46:53.394415 9420 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:53.394562 9420 raft_consensus.cc:482] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.394623 9420 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.395576 9420 raft_consensus.cc:504] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.395895 9420 leader_election.cc:288] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 5 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.396304 9689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 5 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.396436 9689 raft_consensus.cc:3032] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 20:46:53.396800 9689 raft_consensus.cc:729] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.396935 9689 consensus_queue.cc:260] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.397066 9689 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.398042 9284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 5 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.398138 9284 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.398284 9689 raft_consensus.cc:2396] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 6.
I20240502 20:46:53.399102 9284 raft_consensus.cc:2396] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 5 because replica has last-logged OpId of term: 3 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 6.
I20240502 20:46:53.399240 9420 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:53.399338 9420 raft_consensus.cc:482] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.399394 9420 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Advancing to term 5
W20240502 20:46:53.399511 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51672: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.400106 9420 raft_consensus.cc:504] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.400501 9420 leader_election.cc:288] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 5 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.401530 9420 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:53.401674 9420 raft_consensus.cc:482] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.401768 9420 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.402520 9420 raft_consensus.cc:504] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.403160 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 5 candidate_status { last_received { term: 3 index: 13 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.403256 9555 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.405077 9355 leader_election.cc:302] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88a19090795d47c2a1ea1dd979fb62bc; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:53.405148 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 5 candidate_status { last_received { term: 3 index: 13 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.405297 11712 raft_consensus.cc:2726] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20240502 20:46:53.405308 9287 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.405437 9424 raft_consensus.cc:1226] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Rejecting Update request from peer 190c1d3968d949bbbaa27a51a2b91d8e for earlier term 5. Current term is 6. Ops: []
I20240502 20:46:53.405815 9420 leader_election.cc:288] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 6 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:53.405939 10554 consensus_queue.cc:1046] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: INVALID_TERM, Last received: 5.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 20:46:53.406075 10554 raft_consensus.cc:3032] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Stepping down as leader of term 5
I20240502 20:46:53.406127 10554 raft_consensus.cc:729] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.406244 10554 consensus_queue.cc:260] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 5.11, Last appended by leader: 11, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.406396 10554 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.406883 9287 raft_consensus.cc:2454] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 5.
I20240502 20:46:53.408315 9555 raft_consensus.cc:2454] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 5.
I20240502 20:46:53.408491 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 6 candidate_status { last_received { term: 5 index: 10 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.408660 9420 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d92bfee8fabd46a28ffcb7e319879406"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:53.408768 9420 raft_consensus.cc:482] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.408856 9420 raft_consensus.cc:3037] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.409080 9355 leader_election.cc:302] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:53.409226 11712 raft_consensus.cc:2781] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Leader election won for term 5
I20240502 20:46:53.409293 11712 raft_consensus.cc:686] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 5 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.409394 11712 consensus_queue.cc:237] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 3.13, Last appended by leader: 13, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.409646 9420 raft_consensus.cc:504] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.410328 9420 leader_election.cc:288] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 4 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.410658 9689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d92bfee8fabd46a28ffcb7e319879406" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 4 candidate_status { last_received { term: 2 index: 15 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.410748 9689 raft_consensus.cc:3037] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 3 FOLLOWER]: Advancing to term 4
W20240502 20:46:53.410933 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.411543 9689 raft_consensus.cc:2454] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 4.
I20240502 20:46:53.411840 9357 leader_election.cc:302] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:53.412132 11710 raft_consensus.cc:2781] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Leader election won for term 4
I20240502 20:46:53.412205 11710 raft_consensus.cc:686] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.412317 11710 consensus_queue.cc:237] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 2.15, Last appended by leader: 15, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.412248 9139 catalog_manager.cc:5484] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 3 to 5, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 5 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:53.413125 9289 raft_consensus.cc:1226] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Rejecting Update request from peer 64136bd82a30486e9d56fc28da4bd448 for earlier term 4. Current term is 5. Ops: []
I20240502 20:46:53.413966 9420 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:53.414067 9420 raft_consensus.cc:482] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.414131 9420 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.415586 9420 raft_consensus.cc:504] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.415550 9139 catalog_manager.cc:5484] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 2 to 4, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 4 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:53.416131 9420 leader_election.cc:288] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 5 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.417347 9550 raft_consensus.cc:1261] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 3 index: 12. Preceding OpId from leader: term: 5 index: 15. (index mismatch)
I20240502 20:46:53.418323 11710 consensus_queue.cc:1035] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 20:46:53.418813 9554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 6 candidate_status { last_received { term: 5 index: 10 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.419521 9554 raft_consensus.cc:2396] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 6 because replica has last-logged OpId of term: 5 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 10.
I20240502 20:46:53.419778 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d92bfee8fabd46a28ffcb7e319879406" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 4 candidate_status { last_received { term: 2 index: 15 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.419889 9286 raft_consensus.cc:3037] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 3 FOLLOWER]: Advancing to term 4
I20240502 20:46:53.420185 9357 leader_election.cc:302] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f
I20240502 20:46:53.420575 11712 raft_consensus.cc:2726] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20240502 20:46:53.420723 9286 raft_consensus.cc:2454] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 4.
I20240502 20:46:53.422063 9287 raft_consensus.cc:1261] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 3 index: 13. Preceding OpId from leader: term: 5 index: 15. (index mismatch)
I20240502 20:46:53.422153 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 5 candidate_status { last_received { term: 4 index: 12 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.422264 9687 raft_consensus.cc:3032] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 20:46:53.422317 9687 raft_consensus.cc:729] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.422446 9687 consensus_queue.cc:260] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 4.12, Last appended by leader: 12, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.422621 9687 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.422859 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 5 candidate_status { last_received { term: 4 index: 12 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.422998 9289 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.423516 9687 raft_consensus.cc:2454] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 5.
I20240502 20:46:53.423954 9420 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "20698b2531674eefb9df3125a86653ba"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:53.424050 9420 raft_consensus.cc:482] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.424093 9420 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.424336 11712 consensus_queue.cc:1035] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 20:46:53.424731 9357 leader_election.cc:302] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:53.424887 11712 raft_consensus.cc:2781] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Leader election won for term 5
I20240502 20:46:53.424965 11712 raft_consensus.cc:686] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 5 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.425067 11712 consensus_queue.cc:237] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 4.12, Last appended by leader: 12, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.426399 9144 catalog_manager.cc:5484] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 4 to 5, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 5 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:53.426860 9689 raft_consensus.cc:1261] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 5 index: 14. Preceding OpId from leader: term: 6 index: 16. (index mismatch)
I20240502 20:46:53.427201 11712 consensus_queue.cc:1035] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 20:46:53.429761 9554 raft_consensus.cc:1261] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 5 index: 14. Preceding OpId from leader: term: 6 index: 16. (index mismatch)
I20240502 20:46:53.429824 9420 raft_consensus.cc:504] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.430099 11710 consensus_queue.cc:1035] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 20:46:53.430492 9550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20698b2531674eefb9df3125a86653ba" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 6 candidate_status { last_received { term: 3 index: 15 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.430588 9550 raft_consensus.cc:3032] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Stepping down as leader of term 5
I20240502 20:46:53.430631 9550 raft_consensus.cc:729] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.430652 9420 leader_election.cc:288] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 6 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:53.430771 9550 consensus_queue.cc:260] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 5.16, Last appended by leader: 16, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.430925 9550 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.431190 9289 raft_consensus.cc:2454] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 5.
I20240502 20:46:53.431849 9550 raft_consensus.cc:2396] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 6 because replica has last-logged OpId of term: 5 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 15.
I20240502 20:46:53.432133 11558 mvcc.cc:205] Tried to move back new op lower bound from 7023340803749670912 to 7023340803726110720. Current Snapshot: MvccSnapshot[applied={T|T < 7023340803749670912}]
I20240502 20:46:53.432485 9420 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:53.432585 9420 raft_consensus.cc:482] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.432634 9420 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.433368 9420 raft_consensus.cc:504] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.437232 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20698b2531674eefb9df3125a86653ba" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 6 candidate_status { last_received { term: 3 index: 15 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.437327 9286 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.438561 9286 raft_consensus.cc:2454] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 6.
I20240502 20:46:53.440923 9420 leader_election.cc:288] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 6 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:53.441428 9554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 6 candidate_status { last_received { term: 3 index: 15 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.441529 9554 raft_consensus.cc:3032] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Stepping down as leader of term 5
I20240502 20:46:53.441563 9554 raft_consensus.cc:729] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.441854 9554 consensus_queue.cc:260] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 5.16, Last appended by leader: 16, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.441983 9554 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.442889 9554 raft_consensus.cc:2396] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 6 because replica has last-logged OpId of term: 5 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 15.
I20240502 20:46:53.443626 9355 leader_election.cc:302] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e
I20240502 20:46:53.443833 11712 raft_consensus.cc:2781] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Leader election won for term 6
I20240502 20:46:53.443909 11712 raft_consensus.cc:686] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 6 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.443933 9689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 6 candidate_status { last_received { term: 3 index: 15 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.444032 9689 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.444011 11712 consensus_queue.cc:237] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 3.15, Last appended by leader: 15, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.444747 9687 raft_consensus.cc:1261] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 2 index: 14. Preceding OpId from leader: term: 4 index: 17. (index mismatch)
I20240502 20:46:53.447562 11628 consensus_queue.cc:1035] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 20:46:53.448184 9144 catalog_manager.cc:5484] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 5 to 6, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 6 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:53.448877 9689 raft_consensus.cc:2396] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 6 because replica has last-logged OpId of term: 4 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 15.
I20240502 20:46:53.450959 9357 leader_election.cc:302] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:53.451099 11628 raft_consensus.cc:2726] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20240502 20:46:53.451442 9289 raft_consensus.cc:1261] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 2 index: 15. Preceding OpId from leader: term: 4 index: 17. (index mismatch)
I20240502 20:46:53.453086 9420 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:53.453197 9420 raft_consensus.cc:482] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.453241 9420 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.454392 9420 raft_consensus.cc:504] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.456072 11628 consensus_queue.cc:1035] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20240502 20:46:53.463068 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51646: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.464658 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 6 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.464749 9686 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.465571 9686 raft_consensus.cc:2396] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 6 because replica has last-logged OpId of term: 4 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 7.
W20240502 20:46:53.468439 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
W20240502 20:46:53.472677 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.476282 9420 leader_election.cc:288] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 6 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:53.476802 9420 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:53.476898 9420 raft_consensus.cc:482] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.476943 9420 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.477691 9420 raft_consensus.cc:504] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.478183 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 5 candidate_status { last_received { term: 2 index: 10 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.478271 9686 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.479068 9686 raft_consensus.cc:2454] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 5.
I20240502 20:46:53.479569 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 6 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.479667 9555 raft_consensus.cc:3032] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Stepping down as leader of term 5
I20240502 20:46:53.479700 9555 raft_consensus.cc:729] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
W20240502 20:46:53.482098 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51646: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.482620 9555 consensus_queue.cc:260] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.483193 9555 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.484768 9357 leader_election.cc:302] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:53.485482 11628 raft_consensus.cc:2781] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Leader election won for term 5
I20240502 20:46:53.485564 11628 raft_consensus.cc:686] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.485668 11628 consensus_queue.cc:237] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.10, Last appended by leader: 10, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.487186 9555 raft_consensus.cc:2396] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 6 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 7.
I20240502 20:46:53.487455 9357 leader_election.cc:302] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:53.487586 11711 raft_consensus.cc:2726] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20240502 20:46:53.488516 9420 leader_election.cc:288] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 5 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.489902 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 5 candidate_status { last_received { term: 2 index: 10 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.490001 9286 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Advancing to term 5
W20240502 20:46:53.491719 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:53.491834 9286 raft_consensus.cc:2396] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 5 because replica has last-logged OpId of term: 3 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 10.
I20240502 20:46:53.493413 9144 catalog_manager.cc:5484] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 3 to 5, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 5 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
W20240502 20:46:53.499898 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.506984 9289 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:53.507086 9289 raft_consensus.cc:482] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.507133 9289 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.508069 9289 raft_consensus.cc:504] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.508666 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.508762 9686 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.509627 9686 raft_consensus.cc:2454] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 5.
I20240502 20:46:53.509917 9223 leader_election.cc:302] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:53.510066 11739 raft_consensus.cc:2781] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Leader election won for term 5
I20240502 20:46:53.510128 11739 raft_consensus.cc:686] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.510231 11739 consensus_queue.cc:237] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
W20240502 20:46:53.510478 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51646: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.513613 9289 leader_election.cc:288] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 5 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:53.514204 9289 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:53.514307 9289 raft_consensus.cc:482] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.514365 9289 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.514379 9144 catalog_manager.cc:5484] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 4 to 5, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 5 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:53.515096 9289 raft_consensus.cc:504] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.515414 9289 leader_election.cc:288] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 6 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:53.516479 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.516628 9555 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.517619 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.517726 9686 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.517657 9555 raft_consensus.cc:2454] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 6.
I20240502 20:46:53.518011 9223 leader_election.cc:302] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:53.518160 11725 raft_consensus.cc:2781] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Leader election won for term 6
I20240502 20:46:53.518244 11725 raft_consensus.cc:686] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.518349 11725 consensus_queue.cc:237] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.518620 9686 raft_consensus.cc:2396] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20240502 20:46:53.518903 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.519003 9420 raft_consensus.cc:3032] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 LEADER]: Stepping down as leader of term 4
I20240502 20:46:53.519038 9420 raft_consensus.cc:729] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.519178 9420 consensus_queue.cc:260] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.519315 9420 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.520045 9420 raft_consensus.cc:2396] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20240502 20:46:53.520227 9289 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:53.520351 9289 raft_consensus.cc:482] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.520411 9289 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.521119 9289 raft_consensus.cc:504] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
W20240502 20:46:53.521140 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:53.521190 9144 catalog_manager.cc:5484] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 4 to 6, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 6 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:53.521440 9289 leader_election.cc:288] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 7 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:53.522069 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.522168 9555 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.522181 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.522266 9420 raft_consensus.cc:3032] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 6 LEADER]: Stepping down as leader of term 6
I20240502 20:46:53.522298 9420 raft_consensus.cc:729] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.522432 9420 consensus_queue.cc:260] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.522581 9420 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.523002 9555 raft_consensus.cc:2454] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 7.
I20240502 20:46:53.523268 9223 leader_election.cc:302] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:53.523425 9420 raft_consensus.cc:2396] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20240502 20:46:53.523448 11739 raft_consensus.cc:2781] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Leader election won for term 7
I20240502 20:46:53.523517 11739 raft_consensus.cc:686] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.523622 11739 consensus_queue.cc:237] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.524659 9289 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:53.524765 9289 raft_consensus.cc:482] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.524820 9289 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.525542 9289 raft_consensus.cc:504] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.525640 9144 catalog_manager.cc:5484] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 6 to 7, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 7 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:53.525846 9289 leader_election.cc:288] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 6 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:53.526126 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 16 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.526221 9687 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.526436 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 16 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.526546 9555 raft_consensus.cc:3032] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Stepping down as leader of term 5
I20240502 20:46:53.526582 9555 raft_consensus.cc:729] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.527081 9687 raft_consensus.cc:2454] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 6.
I20240502 20:46:53.527261 9555 consensus_queue.cc:260] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 5.16, Last appended by leader: 16, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.527408 9555 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.527539 9223 leader_election.cc:302] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:53.527536 9289 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:53.527710 11739 raft_consensus.cc:2781] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Leader election won for term 6
I20240502 20:46:53.527747 9289 raft_consensus.cc:482] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.527783 11739 raft_consensus.cc:686] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.527936 9289 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.527976 11739 consensus_queue.cc:237] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 5.16, Last appended by leader: 16, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.528163 9555 raft_consensus.cc:2454] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 6.
I20240502 20:46:53.528815 9289 raft_consensus.cc:504] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.529103 9289 leader_election.cc:288] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 5 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:53.529628 9550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 32 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.530031 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 32 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.530129 9686 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.530498 9289 raft_consensus.cc:1226] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Rejecting Update request from peer 190c1d3968d949bbbaa27a51a2b91d8e for earlier term 4. Current term is 5. Ops: [4.33-4.33]
I20240502 20:46:53.530822 11532 consensus_queue.cc:1046] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: INVALID_TERM, Last received: 4.32, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20240502 20:46:53.530984 9686 raft_consensus.cc:2396] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 5 because replica has last-logged OpId of term: 4 index: 33, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 32.
I20240502 20:46:53.530995 11532 raft_consensus.cc:3032] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 LEADER]: Stepping down as leader of term 4
I20240502 20:46:53.531037 11532 raft_consensus.cc:729] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.531142 11532 consensus_queue.cc:260] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 4.33, Last appended by leader: 33, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.531260 11532 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Advancing to term 5
W20240502 20:46:53.531484 11536 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51672: Illegal state: Replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:53.531756 9223 leader_election.cc:302] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:53.532048 11739 raft_consensus.cc:2726] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20240502 20:46:53.532050 9289 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:53.532277 9289 raft_consensus.cc:482] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.532384 9289 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Advancing to term 7
W20240502 20:46:53.532531 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
W20240502 20:46:53.532931 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51610: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.533159 9687 raft_consensus.cc:1226] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Rejecting Update request from peer 190c1d3968d949bbbaa27a51a2b91d8e for earlier term 4. Current term is 5. Ops: []
I20240502 20:46:53.533345 9289 raft_consensus.cc:504] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.533699 9289 leader_election.cc:288] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 7 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:53.533977 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 7 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.534119 9420 raft_consensus.cc:3032] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 6 LEADER]: Stepping down as leader of term 6
I20240502 20:46:53.534206 9420 raft_consensus.cc:729] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.534384 9420 consensus_queue.cc:260] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.11, Last appended by leader: 11, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.534385 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 7 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.534541 9555 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.534646 9420 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.535068 9139 catalog_manager.cc:5484] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 5 to 6, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 6 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:53.535732 9420 raft_consensus.cc:2396] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 7 because replica has last-logged OpId of term: 6 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 10.
W20240502 20:46:53.536098 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59728: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.536499 9289 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:53.536599 9289 raft_consensus.cc:482] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.536648 9289 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.537207 9686 raft_consensus.cc:1261] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 5 index: 16. Preceding OpId from leader: term: 6 index: 18. (index mismatch)
I20240502 20:46:53.537575 11725 consensus_queue.cc:1035] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240502 20:46:53.539155 9289 raft_consensus.cc:504] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.539482 9289 leader_election.cc:288] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 6 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
W20240502 20:46:53.540076 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40118: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.540815 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.540962 9686 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.541229 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.541337 9420 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.542076 9420 raft_consensus.cc:2454] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 6.
I20240502 20:46:53.542399 9222 leader_election.cc:302] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:53.542562 11739 raft_consensus.cc:2781] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Leader election won for term 6
I20240502 20:46:53.542589 9686 raft_consensus.cc:2396] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 7.
I20240502 20:46:53.542636 11739 raft_consensus.cc:686] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.542738 11739 consensus_queue.cc:237] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.543465 9555 raft_consensus.cc:2454] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 7.
I20240502 20:46:53.543737 9223 leader_election.cc:302] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters: 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:53.543978 11725 raft_consensus.cc:2781] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Leader election won for term 7
I20240502 20:46:53.544054 11725 raft_consensus.cc:686] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.544153 11725 consensus_queue.cc:237] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 4.10, Last appended by leader: 10, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.544211 9554 raft_consensus.cc:1261] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 5 index: 16. Preceding OpId from leader: term: 6 index: 18. (index mismatch)
I20240502 20:46:53.544245 9144 catalog_manager.cc:5484] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 4 to 6, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 6 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:53.545315 11739 consensus_queue.cc:1035] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20240502 20:46:53.548801 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51610: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
W20240502 20:46:53.548918 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51646: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.549790 9289 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:53.549890 9289 raft_consensus.cc:482] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.549955 9289 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.550722 9289 raft_consensus.cc:504] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.552667 9136 catalog_manager.cc:5484] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 6 to 7, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 7 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:53.553411 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.553519 9420 raft_consensus.cc:3032] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 6 LEADER]: Stepping down as leader of term 6
I20240502 20:46:53.553570 9420 raft_consensus.cc:729] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.553722 9420 consensus_queue.cc:260] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.553925 9420 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.555006 9420 raft_consensus.cc:2396] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20240502 20:46:53.555945 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59728: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.557090 9289 leader_election.cc:288] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 7 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:53.557247 9554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.557363 9554 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.557633 9287 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3315525f1849462fbf49a285f424ea51"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:53.557727 9287 raft_consensus.cc:482] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.557770 9287 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.558199 9554 raft_consensus.cc:2454] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 7.
I20240502 20:46:53.558584 9223 leader_election.cc:302] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters: 88a19090795d47c2a1ea1dd979fb62bc
W20240502 20:46:53.561086 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:53.558841 9287 raft_consensus.cc:504] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.558977 11739 raft_consensus.cc:2781] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Leader election won for term 7
I20240502 20:46:53.561542 11739 raft_consensus.cc:686] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.561659 11739 consensus_queue.cc:237] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.562911 9287 leader_election.cc:288] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 7 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:53.563586 9142 catalog_manager.cc:5484] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 6 to 7, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 7 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
W20240502 20:46:53.564201 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40118: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.566921 9550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3315525f1849462fbf49a285f424ea51" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 32 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.567040 9550 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.567698 9287 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fe22224142984d69a2def286584b9aaa"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:53.567811 9287 raft_consensus.cc:482] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.567868 9287 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.567950 9550 raft_consensus.cc:2454] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 7.
I20240502 20:46:53.568262 9223 leader_election.cc:302] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:53.568619 11725 raft_consensus.cc:2781] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Leader election won for term 7
I20240502 20:46:53.568706 11725 raft_consensus.cc:686] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.568825 11725 consensus_queue.cc:237] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 6.32, Last appended by leader: 32, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.570106 9287 raft_consensus.cc:504] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.571246 9287 leader_election.cc:288] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 6 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:53.571488 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe22224142984d69a2def286584b9aaa" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 19 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.571592 9686 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.571686 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe22224142984d69a2def286584b9aaa" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 19 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.571774 9424 raft_consensus.cc:3032] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 5 LEADER]: Stepping down as leader of term 5
I20240502 20:46:53.571820 9424 raft_consensus.cc:729] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.571954 9424 consensus_queue.cc:260] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 5.20, Last appended by leader: 20, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.572095 9424 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.572559 9686 raft_consensus.cc:2454] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 6.
I20240502 20:46:53.573038 9424 raft_consensus.cc:2396] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 6 because replica has last-logged OpId of term: 5 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 19.
I20240502 20:46:53.572803 9223 leader_election.cc:302] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:53.573710 11739 raft_consensus.cc:2781] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Leader election won for term 6
I20240502 20:46:53.573804 11739 raft_consensus.cc:686] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.573904 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3315525f1849462fbf49a285f424ea51" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 7 candidate_status { last_received { term: 6 index: 32 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.573921 11739 consensus_queue.cc:237] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 3.19, Last appended by leader: 19, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.574072 9420 raft_consensus.cc:3032] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 LEADER]: Stepping down as leader of term 6
I20240502 20:46:53.574151 9420 raft_consensus.cc:729] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.574311 9420 consensus_queue.cc:260] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 6.32, Last appended by leader: 32, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.574570 9420 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Advancing to term 7
W20240502 20:46:53.575445 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.575594 9287 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:53.575736 9287 raft_consensus.cc:482] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.575937 9287 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.576761 9287 raft_consensus.cc:504] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.577414 9287 leader_election.cc:288] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 6 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:53.577752 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 6 candidate_status { last_received { term: 4 index: 16 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.577863 9687 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.577769 9144 catalog_manager.cc:5484] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 6 to 7, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 7 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:53.578792 9687 raft_consensus.cc:2454] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 6.
I20240502 20:46:53.579661 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 6 candidate_status { last_received { term: 4 index: 16 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.579780 9424 raft_consensus.cc:3032] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 LEADER]: Stepping down as leader of term 5
I20240502 20:46:53.579828 9424 raft_consensus.cc:729] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.579952 9424 consensus_queue.cc:260] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 5.17, Last appended by leader: 17, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.580083 9424 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.580895 9424 raft_consensus.cc:2396] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 6 because replica has last-logged OpId of term: 5 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 16.
W20240502 20:46:53.581122 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51610: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.581179 9223 leader_election.cc:302] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:53.582577 11725 raft_consensus.cc:2781] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Leader election won for term 6
I20240502 20:46:53.582661 11725 raft_consensus.cc:686] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.582777 11725 consensus_queue.cc:237] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 4.16, Last appended by leader: 16, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.582963 9139 catalog_manager.cc:5484] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 5 to 6, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 6 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:53.584717 9287 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:53.584874 9287 raft_consensus.cc:482] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.585327 9287 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.587105 9142 catalog_manager.cc:5484] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 5 to 6, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 6 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:53.587543 9287 raft_consensus.cc:504] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.587872 9287 leader_election.cc:288] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 5 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:53.588225 9550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 5 candidate_status { last_received { term: 2 index: 12 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.588325 9550 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.589157 9550 raft_consensus.cc:2454] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 5.
I20240502 20:46:53.590695 9223 leader_election.cc:302] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:53.590950 11739 raft_consensus.cc:2781] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Leader election won for term 5
I20240502 20:46:53.591041 11739 raft_consensus.cc:686] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
W20240502 20:46:53.591116 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51646: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.591156 11739 consensus_queue.cc:237] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.12, Last appended by leader: 12, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.591373 9420 raft_consensus.cc:2454] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 7.
I20240502 20:46:53.592383 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 5 candidate_status { last_received { term: 2 index: 12 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.592483 9687 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.593119 9139 catalog_manager.cc:5484] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 3 to 5, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 5 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
W20240502 20:46:53.593416 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59728: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.593798 9287 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:53.593899 9287 raft_consensus.cc:482] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.593951 9287 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.594516 9687 raft_consensus.cc:2396] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 5 because replica has last-logged OpId of term: 3 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 12.
I20240502 20:46:53.594753 9287 raft_consensus.cc:504] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.595069 9287 leader_election.cc:288] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 6 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:53.595683 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 19 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.595798 9420 raft_consensus.cc:3032] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 5 LEADER]: Stepping down as leader of term 5
I20240502 20:46:53.595839 9420 raft_consensus.cc:729] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.595973 9420 consensus_queue.cc:260] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 5.19, Last appended by leader: 19, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.596118 9420 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.596887 9420 raft_consensus.cc:2454] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 6.
I20240502 20:46:53.597112 9222 leader_election.cc:302] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:53.597261 11725 raft_consensus.cc:2781] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Leader election won for term 6
I20240502 20:46:53.597347 11725 raft_consensus.cc:686] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.597451 11725 consensus_queue.cc:237] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 5.19, Last appended by leader: 19, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.598433 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 19 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.598555 9555 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.599059 9139 catalog_manager.cc:5484] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 5 to 6, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 6 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:53.599823 9555 raft_consensus.cc:2454] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 6.
W20240502 20:46:53.599936 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55574: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:53.600204 9287 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:53.600299 9287 raft_consensus.cc:482] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.600348 9287 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.601058 9287 raft_consensus.cc:504] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.602383 9287 leader_election.cc:288] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 5 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
W20240502 20:46:53.603663 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40118: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.604811 9287 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:53.604911 9287 raft_consensus.cc:482] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.604959 9287 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.605057 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 5 candidate_status { last_received { term: 2 index: 16 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.605149 9555 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.605916 9287 raft_consensus.cc:504] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.607055 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 5 candidate_status { last_received { term: 2 index: 16 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.607055 9554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 10 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.607180 9554 raft_consensus.cc:2396] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 6 because replica has last-logged OpId of term: 5 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 10.
I20240502 20:46:53.607196 9687 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.607494 9287 leader_election.cc:288] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 6 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:53.605928 9555 raft_consensus.cc:2454] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 5.
W20240502 20:46:53.607874 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51672: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.608089 9223 leader_election.cc:302] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:53.608132 9289 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d92bfee8fabd46a28ffcb7e319879406"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:53.608235 11739 raft_consensus.cc:2781] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Leader election won for term 5
I20240502 20:46:53.608335 11739 raft_consensus.cc:686] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.608450 11739 consensus_queue.cc:237] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.16, Last appended by leader: 16, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.608600 9687 raft_consensus.cc:2396] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 5 because replica has last-logged OpId of term: 3 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 16.
W20240502 20:46:53.609383 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:53.609598 9289 raft_consensus.cc:482] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.609673 9289 raft_consensus.cc:3037] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.610579 9289 raft_consensus.cc:504] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.611610 9289 leader_election.cc:288] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 5 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:53.612064 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d92bfee8fabd46a28ffcb7e319879406" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 18 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.612159 9687 raft_consensus.cc:3037] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.612599 9289 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:53.613150 9687 raft_consensus.cc:2454] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 5.
I20240502 20:46:53.614507 9289 raft_consensus.cc:482] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.614620 9289 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.614853 9142 catalog_manager.cc:5484] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 3 to 5, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 5 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
W20240502 20:46:53.615233 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51610: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.615455 9289 raft_consensus.cc:504] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.615705 9555 raft_consensus.cc:1261] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 5 index: 19. Preceding OpId from leader: term: 6 index: 21. (index mismatch)
I20240502 20:46:53.615736 9289 leader_election.cc:288] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 6 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:53.614387 9223 leader_election.cc:302] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:53.615919 11739 raft_consensus.cc:2781] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Leader election won for term 5
I20240502 20:46:53.615998 11739 raft_consensus.cc:686] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.616101 11739 consensus_queue.cc:237] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 4.18, Last appended by leader: 18, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.616951 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 6 candidate_status { last_received { term: 4 index: 12 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.617046 9687 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.617025 11739 consensus_queue.cc:1035] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20240502 20:46:53.617592 9289 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:53.617691 9289 raft_consensus.cc:482] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.617743 9289 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.618736 9289 raft_consensus.cc:504] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.618831 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d92bfee8fabd46a28ffcb7e319879406" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 5 candidate_status { last_received { term: 4 index: 18 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.618937 9424 raft_consensus.cc:3032] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 LEADER]: Stepping down as leader of term 4
I20240502 20:46:53.618976 9424 raft_consensus.cc:729] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.619089 9424 consensus_queue.cc:260] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 4.18, Last appended by leader: 18, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.619161 9423 raft_consensus.cc:1261] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 5 index: 19. Preceding OpId from leader: term: 6 index: 21. (index mismatch)
I20240502 20:46:53.619241 9424 raft_consensus.cc:3037] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 4 FOLLOWER]: Advancing to term 5
I20240502 20:46:53.619247 9289 leader_election.cc:288] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 6 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:53.619714 9287 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "20698b2531674eefb9df3125a86653ba"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:53.619809 9287 raft_consensus.cc:482] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.619789 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 6 candidate_status { last_received { term: 4 index: 12 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.619855 9287 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.619874 9423 raft_consensus.cc:3032] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 5 LEADER]: Stepping down as leader of term 5
I20240502 20:46:53.619908 9423 raft_consensus.cc:729] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.620018 9423 consensus_queue.cc:260] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 5.13, Last appended by leader: 13, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.620043 9424 raft_consensus.cc:2454] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 5.
I20240502 20:46:53.620134 9423 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.620571 9554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 29 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.620563 9287 raft_consensus.cc:504] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.620656 9554 raft_consensus.cc:3032] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Stepping down as leader of term 5
I20240502 20:46:53.620685 9554 raft_consensus.cc:729] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.620841 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 29 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.620919 9686 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.621071 9423 raft_consensus.cc:2396] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 6 because replica has last-logged OpId of term: 5 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 12.
I20240502 20:46:53.621104 9550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20698b2531674eefb9df3125a86653ba" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 7 candidate_status { last_received { term: 3 index: 14 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.621181 9550 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.621352 9554 consensus_queue.cc:260] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 5.29, Last appended by leader: 29, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.621511 9554 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.621765 11739 consensus_queue.cc:1035] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20240502 20:46:53.622058 9550 raft_consensus.cc:2396] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 7 because replica has last-logged OpId of term: 5 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 14.
I20240502 20:46:53.622450 9287 leader_election.cc:288] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 7 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:53.622673 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20698b2531674eefb9df3125a86653ba" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 7 candidate_status { last_received { term: 3 index: 14 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.622761 9423 raft_consensus.cc:3032] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 6 LEADER]: Stepping down as leader of term 6
I20240502 20:46:53.622810 9423 raft_consensus.cc:729] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.622933 9423 consensus_queue.cc:260] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 6.16, Last appended by leader: 16, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.623000 9286 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:53.623055 9423 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.623081 9286 raft_consensus.cc:482] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.623135 9286 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.623325 9686 raft_consensus.cc:2454] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 6.
I20240502 20:46:53.623675 9554 raft_consensus.cc:2454] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 6.
I20240502 20:46:53.623872 9423 raft_consensus.cc:2396] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 7 because replica has last-logged OpId of term: 6 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 14.
I20240502 20:46:53.623888 9286 raft_consensus.cc:504] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.624167 9286 leader_election.cc:288] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 7 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:53.624265 9687 raft_consensus.cc:2454] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 6.
I20240502 20:46:53.624828 9223 leader_election.cc:302] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:53.624986 11739 raft_consensus.cc:2781] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Leader election won for term 6
I20240502 20:46:53.625072 11739 raft_consensus.cc:686] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.625173 11739 consensus_queue.cc:237] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 5.29, Last appended by leader: 29, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.625509 9223 leader_election.cc:302] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters: 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:53.625651 11739 raft_consensus.cc:2781] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Leader election won for term 6
I20240502 20:46:53.625720 11739 raft_consensus.cc:686] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.625824 11739 consensus_queue.cc:237] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 4.12, Last appended by leader: 12, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.626861 9286 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:53.626821 9136 catalog_manager.cc:5484] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 4 to 5, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 5 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:53.626947 9286 raft_consensus.cc:482] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.626992 9286 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.627756 9286 raft_consensus.cc:504] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.628043 9286 leader_election.cc:288] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 6 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:53.628418 9550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 15 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.628512 9550 raft_consensus.cc:3032] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Stepping down as leader of term 5
I20240502 20:46:53.628548 9550 raft_consensus.cc:729] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.628680 9550 consensus_queue.cc:260] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 5.16, Last appended by leader: 16, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.628800 9550 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Advancing to term 6
W20240502 20:46:53.629364 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59728: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.629626 9550 raft_consensus.cc:2396] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 6 because replica has last-logged OpId of term: 5 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 15.
I20240502 20:46:53.630100 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 15 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.630123 9554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 7 candidate_status { last_received { term: 4 index: 13 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.630199 9687 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.630211 9554 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.630574 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 7 candidate_status { last_received { term: 4 index: 13 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.630666 9423 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.631054 9554 raft_consensus.cc:2396] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 7 because replica has last-logged OpId of term: 5 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 13.
I20240502 20:46:53.631794 9286 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:53.631887 9286 raft_consensus.cc:482] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.631933 9286 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Advancing to term 6
W20240502 20:46:53.632087 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.632570 9423 raft_consensus.cc:2454] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 7.
I20240502 20:46:53.632694 9286 raft_consensus.cc:504] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.632851 9687 raft_consensus.cc:2454] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 6.
I20240502 20:46:53.632972 9286 leader_election.cc:288] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 6 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:53.632972 9222 leader_election.cc:302] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:53.633188 11739 raft_consensus.cc:2726] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20240502 20:46:53.633407 9222 leader_election.cc:302] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:53.633524 11739 raft_consensus.cc:2781] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Leader election won for term 7
I20240502 20:46:53.633592 11739 raft_consensus.cc:686] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.633692 11739 consensus_queue.cc:237] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 4.13, Last appended by leader: 13, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.635049 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 10 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.635145 9686 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.635924 9686 raft_consensus.cc:2454] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 6.
I20240502 20:46:53.635955 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 6 candidate_status { last_received { term: 3 index: 10 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.636036 9423 raft_consensus.cc:3032] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 LEADER]: Stepping down as leader of term 5
I20240502 20:46:53.636070 9423 raft_consensus.cc:729] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.636271 9423 consensus_queue.cc:260] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 5.11, Last appended by leader: 11, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.636346 9223 leader_election.cc:302] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters: 190c1d3968d949bbbaa27a51a2b91d8e
I20240502 20:46:53.636408 9423 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.636524 9223 leader_election.cc:302] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:53.636736 11725 raft_consensus.cc:2781] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Leader election won for term 6
I20240502 20:46:53.636811 11725 raft_consensus.cc:686] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.636811 11739 raft_consensus.cc:2781] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Leader election won for term 6
I20240502 20:46:53.636942 11739 raft_consensus.cc:686] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.636922 11725 consensus_queue.cc:237] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 3.15, Last appended by leader: 15, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.637044 11739 consensus_queue.cc:237] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 3.10, Last appended by leader: 10, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.637712 9142 catalog_manager.cc:5484] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 5 to 6, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 6 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:53.637928 9142 catalog_manager.cc:5484] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 5 to 7, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 7 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:53.638093 9142 catalog_manager.cc:5484] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 5 to 6, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 6 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:53.639021 9423 raft_consensus.cc:2396] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 6 because replica has last-logged OpId of term: 5 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 10.
W20240502 20:46:53.641491 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40118: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.642727 9686 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.642840 9686 raft_consensus.cc:482] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.642899 9686 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.643599 9142 catalog_manager.cc:5484] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 5 to 6, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 6 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:53.643729 9686 raft_consensus.cc:504] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.643908 9142 catalog_manager.cc:5484] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 5 to 6, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 6 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:53.644423 9686 leader_election.cc:288] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 7 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.645159 9689 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.645255 9689 raft_consensus.cc:482] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.645314 9689 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.646109 9689 raft_consensus.cc:504] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.646495 9689 leader_election.cc:288] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 6 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.647563 9689 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.647663 9689 raft_consensus.cc:482] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.647729 9689 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.648474 9689 raft_consensus.cc:504] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.648787 9689 leader_election.cc:288] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 7 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:53.648753 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.648865 9287 raft_consensus.cc:3032] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 LEADER]: Stepping down as leader of term 5
I20240502 20:46:53.648905 9287 raft_consensus.cc:729] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.649022 9287 consensus_queue.cc:260] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.649155 9287 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.649444 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 7 candidate_status { last_received { term: 6 index: 27 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.649534 9555 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Advancing to term 7
W20240502 20:46:53.649574 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51646: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.649995 9287 raft_consensus.cc:2396] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20240502 20:46:53.650249 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 6 candidate_status { last_received { term: 5 index: 10 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.650385 9555 raft_consensus.cc:2454] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 7.
I20240502 20:46:53.650521 9420 raft_consensus.cc:2379] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f in current term 6: Already voted for candidate 88a19090795d47c2a1ea1dd979fb62bc in this term.
I20240502 20:46:53.650812 9222 leader_election.cc:302] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:53.651118 9623 leader_election.cc:302] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:53.651149 10895 raft_consensus.cc:2726] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20240502 20:46:53.651294 11758 raft_consensus.cc:2781] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 20:46:53.651371 11758 raft_consensus.cc:686] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 7 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.651495 11758 consensus_queue.cc:237] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 6.27, Last appended by leader: 27, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.652846 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 7 candidate_status { last_received { term: 6 index: 27 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.652962 9424 raft_consensus.cc:3032] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 LEADER]: Stepping down as leader of term 6
I20240502 20:46:53.653007 9424 raft_consensus.cc:729] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.653142 9424 consensus_queue.cc:260] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 6.27, Last appended by leader: 27, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.653323 9424 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.653589 9142 catalog_manager.cc:5484] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 6 to 7, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 7 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:53.654197 9424 raft_consensus.cc:2454] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 7.
I20240502 20:46:53.654596 9689 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.654690 9689 raft_consensus.cc:482] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.654737 9689 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.655472 9689 raft_consensus.cc:504] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.655859 9689 leader_election.cc:288] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 7 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.656149 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 7 candidate_status { last_received { term: 6 index: 20 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.656261 9287 raft_consensus.cc:3032] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Stepping down as leader of term 6
I20240502 20:46:53.656319 9287 raft_consensus.cc:729] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.656549 9689 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.656656 9689 raft_consensus.cc:482] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.656715 9689 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.657441 9689 raft_consensus.cc:504] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.657579 9667 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:46:53.657963 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 6 candidate_status { last_received { term: 4 index: 33 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.657989 9287 consensus_queue.cc:260] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 6.23, Last appended by leader: 23, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.658068 9555 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.658353 9287 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.658933 9555 raft_consensus.cc:2454] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 6.
I20240502 20:46:53.659219 9287 raft_consensus.cc:2396] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 7 because replica has last-logged OpId of term: 6 index: 23, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 20.
I20240502 20:46:53.659245 9623 leader_election.cc:302] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:53.659411 11624 raft_consensus.cc:2781] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Leader election won for term 6
I20240502 20:46:53.659488 11624 raft_consensus.cc:686] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 6 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.659610 11624 consensus_queue.cc:237] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 4.33, Last appended by leader: 33, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.660362 9689 leader_election.cc:288] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 6 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.660912 9550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 7 candidate_status { last_received { term: 6 index: 20 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.661021 9550 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.661881 9550 raft_consensus.cc:2396] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 7 because replica has last-logged OpId of term: 6 index: 23, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 20.
I20240502 20:46:53.662159 9623 leader_election.cc:302] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f
I20240502 20:46:53.662369 11758 raft_consensus.cc:2726] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20240502 20:46:53.662575 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.662678 9423 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.663895 9423 raft_consensus.cc:2396] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20240502 20:46:53.663861 9688 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.663969 9688 raft_consensus.cc:482] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.664011 9688 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Advancing to term 7
W20240502 20:46:53.664541 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59778: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.664742 9688 raft_consensus.cc:504] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.664929 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 6 candidate_status { last_received { term: 4 index: 33 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.665069 9688 leader_election.cc:288] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 7 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:53.665043 9287 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.665197 9554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.665305 9554 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.665521 9622 leader_election.cc:302] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:53.665709 11758 raft_consensus.cc:2726] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20240502 20:46:53.666045 9144 catalog_manager.cc:5484] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 4 to 6, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 6 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:53.666244 9554 raft_consensus.cc:2454] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 7.
I20240502 20:46:53.666370 9688 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.666499 9688 raft_consensus.cc:482] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.666575 9623 leader_election.cc:302] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:53.666776 11758 raft_consensus.cc:2781] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 20:46:53.666855 11758 raft_consensus.cc:686] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 7 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.666975 11758 consensus_queue.cc:237] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.667632 9287 raft_consensus.cc:2454] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 6.
I20240502 20:46:53.666566 9688 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.665037 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.667791 9289 raft_consensus.cc:3032] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Stepping down as leader of term 6
I20240502 20:46:53.667834 9289 raft_consensus.cc:729] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.667950 9289 consensus_queue.cc:260] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.668066 9289 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.668556 9688 raft_consensus.cc:504] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.668882 9289 raft_consensus.cc:2396] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20240502 20:46:53.669206 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:53.668841 9688 leader_election.cc:288] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 6 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:53.670265 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.670367 9555 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.671072 9144 catalog_manager.cc:5484] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 6 to 7, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 7 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:53.671396 9555 raft_consensus.cc:2396] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20240502 20:46:53.671734 9688 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
W20240502 20:46:53.671761 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51672: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.671825 9688 raft_consensus.cc:482] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.671869 9688 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.672780 9688 raft_consensus.cc:504] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.673219 9688 leader_election.cc:288] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 7 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.673624 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.673777 9424 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.674142 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.674278 9423 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.674787 9424 raft_consensus.cc:2454] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 7.
I20240502 20:46:53.675141 9622 leader_election.cc:302] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:53.675185 9423 raft_consensus.cc:2454] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 6.
I20240502 20:46:53.675562 11624 raft_consensus.cc:2781] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 20:46:53.675652 11624 raft_consensus.cc:686] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 7 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.675781 11624 consensus_queue.cc:237] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.677104 9622 leader_election.cc:302] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e
I20240502 20:46:53.677255 11758 raft_consensus.cc:2781] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Leader election won for term 6
I20240502 20:46:53.677335 11758 raft_consensus.cc:686] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 6 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.677429 11758 consensus_queue.cc:237] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
W20240502 20:46:53.678056 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40138: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.679160 9142 catalog_manager.cc:5484] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 6 to 7, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 7 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:53.679955 9286 raft_consensus.cc:1261] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 6 index: 35. (index mismatch)
I20240502 20:46:53.680338 11758 consensus_queue.cc:1035] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 32, Time since last communication: 0.000s
I20240502 20:46:53.678069 9688 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fe22224142984d69a2def286584b9aaa"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.681193 9688 raft_consensus.cc:482] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.681291 9688 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.681454 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.681556 9287 raft_consensus.cc:3032] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Stepping down as leader of term 6
I20240502 20:46:53.681602 9287 raft_consensus.cc:729] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.681725 9287 consensus_queue.cc:260] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.681847 9287 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.682495 9144 catalog_manager.cc:5484] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 4 to 6, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 6 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:53.682606 9287 raft_consensus.cc:2396] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20240502 20:46:53.682562 9688 raft_consensus.cc:504] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.683307 9688 leader_election.cc:288] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 7 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:53.683668 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe22224142984d69a2def286584b9aaa" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 7 candidate_status { last_received { term: 2 index: 18 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.683952 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe22224142984d69a2def286584b9aaa" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 7 candidate_status { last_received { term: 2 index: 18 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.684000 9555 raft_consensus.cc:1261] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 6 index: 35. (index mismatch)
I20240502 20:46:53.684044 9287 raft_consensus.cc:3032] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Stepping down as leader of term 6
I20240502 20:46:53.684082 9287 raft_consensus.cc:729] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.684191 9287 consensus_queue.cc:260] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 6.20, Last appended by leader: 20, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.684329 9287 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.685098 9287 raft_consensus.cc:2396] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 7 because replica has last-logged OpId of term: 6 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 18.
I20240502 20:46:53.685485 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 7 candidate_status { last_received { term: 5 index: 29 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.685576 9289 raft_consensus.cc:3032] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Stepping down as leader of term 6
I20240502 20:46:53.685613 9289 raft_consensus.cc:729] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.685734 9289 consensus_queue.cc:260] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 6.30, Last appended by leader: 30, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.685850 9289 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.686307 9688 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.686409 9688 raft_consensus.cc:482] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.686487 9688 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Advancing to term 6
W20240502 20:46:53.686527 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59778: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.687609 9289 raft_consensus.cc:2396] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 7 because replica has last-logged OpId of term: 6 index: 30, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 29.
I20240502 20:46:53.688172 9688 raft_consensus.cc:504] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.688921 9688 leader_election.cc:288] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 6 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:53.689105 9550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 7 candidate_status { last_received { term: 5 index: 29 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.689199 9550 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.690052 9550 raft_consensus.cc:2454] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 7.
W20240502 20:46:53.691287 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
W20240502 20:46:53.693537 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59728: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.694389 9554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 6 candidate_status { last_received { term: 3 index: 11 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.694525 9554 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.694972 9623 leader_election.cc:302] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters: 4b3cb6ad00954dc5a1270dc392870b4f
W20240502 20:46:53.695055 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51672: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.695116 11758 consensus_queue.cc:1035] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20240502 20:46:53.695509 9554 raft_consensus.cc:2396] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 6 because replica has last-logged OpId of term: 4 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 11.
I20240502 20:46:53.695513 11031 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:53.696012 11758 raft_consensus.cc:2781] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 20:46:53.696102 11758 raft_consensus.cc:686] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 7 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.696210 11758 consensus_queue.cc:237] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 5.29, Last appended by leader: 29, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.697114 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "edebc185b24441da9504c78394378f4d"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.697203 9687 raft_consensus.cc:482] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.697880 9136 catalog_manager.cc:5484] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 6 to 7, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 7 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
W20240502 20:46:53.699599 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51610: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.699730 9687 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.701005 9687 raft_consensus.cc:504] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.703125 9550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edebc185b24441da9504c78394378f4d" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 7 candidate_status { last_received { term: 4 index: 13 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.703227 9550 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.704067 9550 raft_consensus.cc:2396] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 7 because replica has last-logged OpId of term: 5 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 13.
W20240502 20:46:53.705076 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40138: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.704460 9687 leader_election.cc:288] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 7 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:53.688764 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 6 candidate_status { last_received { term: 3 index: 11 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.691645 9404 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:46:53.702828 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edebc185b24441da9504c78394378f4d" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 7 candidate_status { last_received { term: 4 index: 13 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.705922 9424 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.705978 9420 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.706835 9420 raft_consensus.cc:2454] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 7.
I20240502 20:46:53.706923 9424 raft_consensus.cc:2454] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 6.
I20240502 20:46:53.707188 9622 leader_election.cc:302] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e
I20240502 20:46:53.707398 11758 raft_consensus.cc:2781] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 20:46:53.707459 9622 leader_election.cc:302] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e
I20240502 20:46:53.707731 11624 raft_consensus.cc:2781] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Leader election won for term 6
I20240502 20:46:53.707798 11624 raft_consensus.cc:686] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 6 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.707891 11624 consensus_queue.cc:237] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 3.11, Last appended by leader: 11, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.707998 9550 raft_consensus.cc:1261] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 5 index: 29. Preceding OpId from leader: term: 7 index: 31. (index mismatch)
I20240502 20:46:53.708405 9289 raft_consensus.cc:1261] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 6 index: 30. Preceding OpId from leader: term: 7 index: 31. (index mismatch)
I20240502 20:46:53.708707 11624 consensus_queue.cc:1035] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20240502 20:46:53.707473 11758 raft_consensus.cc:686] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 7 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.708817 11624 consensus_queue.cc:1225] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Peer 4b3cb6ad00954dc5a1270dc392870b4f log is divergent from this leader: its last log entry 6.30 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 29
I20240502 20:46:53.708945 11758 consensus_queue.cc:237] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 4.13, Last appended by leader: 13, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.709453 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.709548 9287 pending_rounds.cc:76] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f: Aborting all ops after (but not including) 29
I20240502 20:46:53.709596 9287 pending_rounds.cc:97] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f: Aborting uncommitted NO_OP operation due to leader change: 6.30
I20240502 20:46:53.709627 9287 raft_consensus.cc:2866] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 20:46:53.709659 9687 raft_consensus.cc:482] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.709808 9687 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.711062 9687 raft_consensus.cc:504] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
W20240502 20:46:53.711652 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51646: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.712023 9142 catalog_manager.cc:5484] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 5 to 7, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 7 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:53.712225 9142 catalog_manager.cc:5484] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 4 to 6, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 6 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:53.712275 9423 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.713321 9423 raft_consensus.cc:2396] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 7 because replica has last-logged OpId of term: 5 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 18.
I20240502 20:46:53.713135 11564 mvcc.cc:205] Tried to move back new op lower bound from 7023340804942835712 to 7023340804901326848. Current Snapshot: MvccSnapshot[applied={T|T < 7023340804942835712}]
I20240502 20:46:53.714699 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 7 candidate_status { last_received { term: 4 index: 16 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.715150 11758 consensus_queue.cc:1035] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
W20240502 20:46:53.718827 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59778: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.721117 9622 leader_election.cc:302] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:53.721355 11624 raft_consensus.cc:2726] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20240502 20:46:53.732072 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51672: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.714809 9289 raft_consensus.cc:3032] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Stepping down as leader of term 6
I20240502 20:46:53.733335 9289 raft_consensus.cc:729] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.733484 9289 consensus_queue.cc:260] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 6.17, Last appended by leader: 17, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.733620 9289 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Advancing to term 7
W20240502 20:46:53.733826 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:53.734114 9423 raft_consensus.cc:1261] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 6 index: 32. Preceding OpId from leader: term: 7 index: 34. (index mismatch)
I20240502 20:46:53.734570 11739 consensus_queue.cc:1035] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20240502 20:46:53.734663 9289 raft_consensus.cc:2396] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 7 because replica has last-logged OpId of term: 6 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 16.
I20240502 20:46:53.735316 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 7 candidate_status { last_received { term: 4 index: 16 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.735409 9420 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.736299 9420 raft_consensus.cc:2396] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 7 because replica has last-logged OpId of term: 5 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 16.
I20240502 20:46:53.737118 9622 leader_election.cc:302] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:53.737390 11758 raft_consensus.cc:2726] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20240502 20:46:53.738794 9687 leader_election.cc:288] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 7 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:53.740142 11300 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:53.741061 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.741173 9687 raft_consensus.cc:482] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.741235 9687 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.742113 9687 raft_consensus.cc:504] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.742692 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 6 candidate_status { last_received { term: 3 index: 16 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.742806 9289 raft_consensus.cc:3032] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 LEADER]: Stepping down as leader of term 5
I20240502 20:46:53.742858 9289 raft_consensus.cc:729] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.742982 9289 consensus_queue.cc:260] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 5.17, Last appended by leader: 17, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.743127 9289 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.744364 9289 raft_consensus.cc:2396] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 6 because replica has last-logged OpId of term: 5 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 16.
I20240502 20:46:53.744721 9687 leader_election.cc:288] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 6 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
W20240502 20:46:53.745903 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40138: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.746378 9550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 6 candidate_status { last_received { term: 3 index: 16 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.746439 9536 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:53.746501 9550 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.747387 9550 raft_consensus.cc:2454] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 6.
I20240502 20:46:53.747748 9623 leader_election.cc:302] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters: 4b3cb6ad00954dc5a1270dc392870b4f
I20240502 20:46:53.747891 11758 raft_consensus.cc:2781] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Leader election won for term 6
I20240502 20:46:53.747957 11758 raft_consensus.cc:686] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 6 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.748060 11758 consensus_queue.cc:237] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 3.16, Last appended by leader: 16, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.748450 9554 raft_consensus.cc:1261] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 6 index: 32. Preceding OpId from leader: term: 7 index: 34. (index mismatch)
I20240502 20:46:53.748802 11739 consensus_queue.cc:1035] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20240502 20:46:53.749665 9136 catalog_manager.cc:5484] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 5 to 6, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 6 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:53.751344 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.751447 9687 raft_consensus.cc:482] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.751497 9687 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.752318 9687 raft_consensus.cc:504] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.752607 9687 leader_election.cc:288] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 6 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.753058 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 6 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.753170 9287 raft_consensus.cc:3032] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 LEADER]: Stepping down as leader of term 5
I20240502 20:46:53.753212 9287 raft_consensus.cc:729] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.753321 9287 consensus_queue.cc:260] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 5.13, Last appended by leader: 13, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.753451 9287 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.754309 9287 raft_consensus.cc:2396] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 6 because replica has last-logged OpId of term: 5 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 12.
W20240502 20:46:53.759088 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
W20240502 20:46:53.764071 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59778: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.768165 9554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 6 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.768288 9554 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.770381 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d92bfee8fabd46a28ffcb7e319879406"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.770524 9687 raft_consensus.cc:482] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.770824 9554 raft_consensus.cc:2454] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 6.
I20240502 20:46:53.772578 9687 raft_consensus.cc:3037] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.771852 9623 leader_election.cc:302] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters: 4b3cb6ad00954dc5a1270dc392870b4f
I20240502 20:46:53.773802 11839 raft_consensus.cc:2781] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Leader election won for term 6
I20240502 20:46:53.773893 11839 raft_consensus.cc:686] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 6 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.774034 11839 consensus_queue.cc:237] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 3.12, Last appended by leader: 12, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.776137 11164 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:53.776482 9136 catalog_manager.cc:5484] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 5 to 6, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 6 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
W20240502 20:46:53.777158 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51672: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.773483 9687 raft_consensus.cc:504] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.780800 9687 leader_election.cc:288] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 6 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.781411 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d92bfee8fabd46a28ffcb7e319879406" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 6 candidate_status { last_received { term: 4 index: 18 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.781517 9423 raft_consensus.cc:3037] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.782374 9423 raft_consensus.cc:2454] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 6.
I20240502 20:46:53.782721 9622 leader_election.cc:302] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:53.782923 11436 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:53.783416 11839 raft_consensus.cc:2781] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Leader election won for term 6
I20240502 20:46:53.783496 11839 raft_consensus.cc:686] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 6 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.783511 9269 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 20:46:53.783596 11839 consensus_queue.cc:237] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 4.18, Last appended by leader: 18, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
W20240502 20:46:53.785454 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51646: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.786794 9142 catalog_manager.cc:5484] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 5 to 6, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 6 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
W20240502 20:46:53.791419 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40138: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.793947 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.794057 9687 raft_consensus.cc:482] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.794332 9687 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.795573 9687 raft_consensus.cc:504] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.796262 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 7 candidate_status { last_received { term: 4 index: 12 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.796380 9286 raft_consensus.cc:3032] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Stepping down as leader of term 6
I20240502 20:46:53.796418 9286 raft_consensus.cc:729] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.796550 9286 consensus_queue.cc:260] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 6.13, Last appended by leader: 13, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.796670 9286 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.797660 9286 raft_consensus.cc:2396] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 7 because replica has last-logged OpId of term: 6 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 12.
I20240502 20:46:53.797899 9687 leader_election.cc:288] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 7 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.803558 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 7 candidate_status { last_received { term: 4 index: 12 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.803668 9423 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.804374 9423 raft_consensus.cc:2396] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 7 because replica has last-logged OpId of term: 5 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 12.
I20240502 20:46:53.804641 9622 leader_election.cc:302] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:53.804841 11838 raft_consensus.cc:2726] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20240502 20:46:53.808740 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59778: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.809334 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d92bfee8fabd46a28ffcb7e319879406" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 6 candidate_status { last_received { term: 4 index: 18 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.809439 9289 raft_consensus.cc:3032] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 LEADER]: Stepping down as leader of term 5
I20240502 20:46:53.809478 9289 raft_consensus.cc:729] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.812237 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.812336 9687 raft_consensus.cc:482] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.812379 9687 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 5 FOLLOWER]: Advancing to term 6
W20240502 20:46:53.812948 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:53.813210 9687 raft_consensus.cc:504] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.813506 9687 leader_election.cc:288] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 6 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:53.813905 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 6 candidate_status { last_received { term: 5 index: 13 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.813972 9554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 6 candidate_status { last_received { term: 5 index: 13 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.814013 9423 raft_consensus.cc:3032] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 LEADER]: Stepping down as leader of term 5
I20240502 20:46:53.814054 9423 raft_consensus.cc:729] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:53.814070 9554 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.814181 9423 consensus_queue.cc:260] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 5.13, Last appended by leader: 13, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.814330 9423 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.814924 9554 raft_consensus.cc:2454] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 6.
I20240502 20:46:53.815026 9423 raft_consensus.cc:2454] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 6.
I20240502 20:46:53.815192 9289 consensus_queue.cc:260] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 5.19, Last appended by leader: 19, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.815343 9289 raft_consensus.cc:3037] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 5 FOLLOWER]: Advancing to term 6
I20240502 20:46:53.815960 9622 leader_election.cc:302] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:53.816220 9289 raft_consensus.cc:2396] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 6 because replica has last-logged OpId of term: 5 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 18.
I20240502 20:46:53.816375 11624 raft_consensus.cc:2781] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Leader election won for term 6
I20240502 20:46:53.816455 11624 raft_consensus.cc:686] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 6 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.816565 11624 consensus_queue.cc:237] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 5.13, Last appended by leader: 13, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.818576 9142 catalog_manager.cc:5484] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 5 to 6, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 6 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:53.821048 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.821185 9687 raft_consensus.cc:482] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.821261 9687 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.822042 9687 raft_consensus.cc:504] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.822403 9687 leader_election.cc:288] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 7 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:53.822753 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.822851 9423 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.823217 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.823326 9555 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.823638 9423 raft_consensus.cc:2454] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 7.
I20240502 20:46:53.824247 9555 raft_consensus.cc:2396] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20240502 20:46:53.824599 9622 leader_election.cc:302] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
W20240502 20:46:53.824762 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51672: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.824963 11838 raft_consensus.cc:2781] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 20:46:53.825039 11838 raft_consensus.cc:686] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 7 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.825142 11838 consensus_queue.cc:237] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.826244 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.826344 9687 raft_consensus.cc:482] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.826400 9687 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.826858 9142 catalog_manager.cc:5484] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 5 to 7, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 7 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:53.827239 9687 raft_consensus.cc:504] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.828501 9687 leader_election.cc:288] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 7 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.829232 9554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 7 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.829315 9554 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.830114 9554 raft_consensus.cc:2396] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 7 because replica has last-logged OpId of term: 5 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 14.
I20240502 20:46:53.834574 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 7 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.834679 9286 raft_consensus.cc:3032] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Stepping down as leader of term 6
I20240502 20:46:53.834715 9286 raft_consensus.cc:729] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.834844 9286 consensus_queue.cc:260] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 6.16, Last appended by leader: 16, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.834966 9286 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.835963 9286 raft_consensus.cc:2396] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 7 because replica has last-logged OpId of term: 6 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 14.
I20240502 20:46:53.836249 9621 leader_election.cc:302] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f
I20240502 20:46:53.836932 11839 raft_consensus.cc:2726] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20240502 20:46:53.838232 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.838330 9687 raft_consensus.cc:482] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.838382 9687 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.839154 9687 raft_consensus.cc:504] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.839444 9687 leader_election.cc:288] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 7 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
W20240502 20:46:53.840081 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.840348 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 7 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.840448 9423 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.841282 9423 raft_consensus.cc:2396] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 7 because replica has last-logged OpId of term: 5 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 9.
I20240502 20:46:53.842020 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 7 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.842126 9289 raft_consensus.cc:3032] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Stepping down as leader of term 6
I20240502 20:46:53.842171 9289 raft_consensus.cc:729] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.842291 9289 consensus_queue.cc:260] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.11, Last appended by leader: 11, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.842413 9289 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.843320 9289 raft_consensus.cc:2396] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 7 because replica has last-logged OpId of term: 6 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 9.
I20240502 20:46:53.843712 9621 leader_election.cc:302] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:53.843909 11758 raft_consensus.cc:2726] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20240502 20:46:53.844228 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40138: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.845080 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.845187 9687 raft_consensus.cc:482] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.845248 9687 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.846033 9687 raft_consensus.cc:504] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.846917 9687 leader_election.cc:288] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 7 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:53.847271 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 7 candidate_status { last_received { term: 4 index: 16 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.847388 9424 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.848253 9424 raft_consensus.cc:2454] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 7.
I20240502 20:46:53.849102 9622 leader_election.cc:302] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:53.849325 11758 raft_consensus.cc:2781] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 20:46:53.849402 11758 raft_consensus.cc:686] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 7 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.849505 11758 consensus_queue.cc:237] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 4.16, Last appended by leader: 16, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.855810 9142 catalog_manager.cc:5484] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 5 to 7, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 7 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:53.858075 9554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 7 candidate_status { last_received { term: 4 index: 16 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.858167 9554 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.859011 9554 raft_consensus.cc:2396] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 7 because replica has last-logged OpId of term: 5 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 16.
W20240502 20:46:53.861862 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59778: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.863750 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:53.863859 9687 raft_consensus.cc:482] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.863909 9687 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.864636 9687 raft_consensus.cc:504] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.865177 9687 leader_election.cc:288] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 7 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:53.865236 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 7 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.865341 9423 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.865849 9550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 7 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:53.865948 9550 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.866199 9423 raft_consensus.cc:2454] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 7.
I20240502 20:46:53.866791 9550 raft_consensus.cc:2396] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 8.
I20240502 20:46:53.868412 9622 leader_election.cc:302] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:53.868893 11839 raft_consensus.cc:2781] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 20:46:53.868973 11839 raft_consensus.cc:686] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 7 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.869081 11839 consensus_queue.cc:237] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.872534 9423 raft_consensus.cc:1261] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 3 index: 15. Preceding OpId from leader: term: 7 index: 18. (index mismatch)
I20240502 20:46:53.874564 11839 consensus_queue.cc:1035] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20240502 20:46:53.874693 9554 raft_consensus.cc:1261] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 5 index: 16. Preceding OpId from leader: term: 7 index: 18. (index mismatch)
I20240502 20:46:53.876067 9554 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.876169 9554 raft_consensus.cc:482] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.876096 9142 catalog_manager.cc:5484] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 5 to 7, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 7 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:53.876214 9554 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.877030 9554 raft_consensus.cc:504] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.877374 9554 leader_election.cc:288] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 7 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:53.878134 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.878237 9424 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.878970 9554 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.879070 9554 raft_consensus.cc:482] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.879118 9554 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:53.879173 9424 raft_consensus.cc:2454] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 7.
I20240502 20:46:53.880021 9491 leader_election.cc:302] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
W20240502 20:46:53.880131 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51672: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.880194 11532 raft_consensus.cc:2781] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Leader election won for term 7
I20240502 20:46:53.880280 11532 raft_consensus.cc:686] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.880393 11532 consensus_queue.cc:237] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.879853 9554 raft_consensus.cc:504] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.881428 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.881522 9688 raft_consensus.cc:3032] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 20:46:53.881559 9688 raft_consensus.cc:729] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.881678 9688 consensus_queue.cc:260] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.881793 9688 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:53.882303 9142 catalog_manager.cc:5484] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 6 to 7, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 7 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:53.882697 9688 raft_consensus.cc:2396] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20240502 20:46:53.883100 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:53.883867 11838 consensus_queue.cc:1035] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20240502 20:46:53.883965 11838 consensus_queue.cc:1225] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Peer 190c1d3968d949bbbaa27a51a2b91d8e log is divergent from this leader: its last log entry 5.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20240502 20:46:53.884577 9550 pending_rounds.cc:76] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e: Aborting all ops after (but not including) 15
I20240502 20:46:53.884621 9550 pending_rounds.cc:97] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e: Aborting uncommitted NO_OP operation due to leader change: 5.16
I20240502 20:46:53.884650 9550 raft_consensus.cc:2866] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 20:46:53.885012 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.885118 9289 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:53.886116 9289 raft_consensus.cc:2396] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20240502 20:46:53.886404 9490 leader_election.cc:302] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:53.886272 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.886536 9687 raft_consensus.cc:3032] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 6 LEADER]: Stepping down as leader of term 6
I20240502 20:46:53.886577 9687 raft_consensus.cc:729] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.886688 9687 consensus_queue.cc:260] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.886807 9687 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.886956 9554 leader_election.cc:288] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.887794 9687 raft_consensus.cc:2396] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20240502 20:46:53.888535 11532 raft_consensus.cc:2726] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20240502 20:46:53.889997 9286 raft_consensus.cc:1261] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 5 index: 19. Preceding OpId from leader: term: 6 index: 20. (index mismatch)
I20240502 20:46:53.890285 9554 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.890393 9554 raft_consensus.cc:482] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.890453 9554 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:53.891273 9554 raft_consensus.cc:504] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.891292 11839 consensus_queue.cc:1035] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20240502 20:46:53.891381 11839 consensus_queue.cc:1225] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Peer 4b3cb6ad00954dc5a1270dc392870b4f log is divergent from this leader: its last log entry 5.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20240502 20:46:53.892220 9286 pending_rounds.cc:76] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f: Aborting all ops after (but not including) 18
I20240502 20:46:53.892289 9286 pending_rounds.cc:97] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f: Aborting uncommitted NO_OP operation due to leader change: 5.19
I20240502 20:46:53.892323 9286 raft_consensus.cc:2866] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 20:46:53.892722 9424 raft_consensus.cc:1261] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 4 index: 18. Preceding OpId from leader: term: 6 index: 20. (index mismatch)
I20240502 20:46:53.893416 11838 consensus_queue.cc:1035] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
W20240502 20:46:53.899184 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59752: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
W20240502 20:46:53.901508 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.903128 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.903215 9289 raft_consensus.cc:3032] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 LEADER]: Stepping down as leader of term 7
I20240502 20:46:53.903023 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.903250 9289 raft_consensus.cc:729] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.903288 9424 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:53.903374 9289 consensus_queue.cc:260] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.903507 9289 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:53.903993 11559 mvcc.cc:205] Tried to move back new op lower bound from 7023340805690126336 to 7023340805258760192. Current Snapshot: MvccSnapshot[applied={T|T < 7023340805690126336 or (T in {7023340805690126336})}]
I20240502 20:46:53.904310 9424 raft_consensus.cc:2396] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 20:46:53.904543 9289 raft_consensus.cc:2396] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 20:46:53.905210 9490 leader_election.cc:302] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:53.905366 11532 raft_consensus.cc:2726] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20240502 20:46:53.905472 9554 leader_election.cc:288] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.906049 9554 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.906149 9554 raft_consensus.cc:482] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.906204 9554 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Advancing to term 8
W20240502 20:46:53.907181 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:53.907181 9554 raft_consensus.cc:504] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.907788 9554 leader_election.cc:288] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:53.907788 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 6 index: 27 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.907888 9687 raft_consensus.cc:3032] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 20:46:53.907925 9687 raft_consensus.cc:729] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
W20240502 20:46:53.908035 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40138: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.908046 9687 consensus_queue.cc:260] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 7.28, Last appended by leader: 28, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.908208 9687 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:53.908859 9554 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.908996 9554 raft_consensus.cc:482] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.909054 9554 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:53.909091 9687 raft_consensus.cc:2396] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 8 because replica has last-logged OpId of term: 7 index: 28, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 27.
I20240502 20:46:53.909659 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 6 index: 27 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.909759 9424 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:53.909796 9554 raft_consensus.cc:504] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.910331 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 6 index: 23 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.910427 9286 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:53.910573 9424 raft_consensus.cc:2454] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 8.
I20240502 20:46:53.910802 9491 leader_election.cc:302] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:53.911370 9286 raft_consensus.cc:2454] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 8.
I20240502 20:46:53.911640 11532 raft_consensus.cc:2781] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Leader election won for term 8
I20240502 20:46:53.911727 11532 raft_consensus.cc:686] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.911854 11532 consensus_queue.cc:237] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 6.27, Last appended by leader: 27, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.913221 9490 leader_election.cc:302] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:53.913376 11578 raft_consensus.cc:2781] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Leader election won for term 8
I20240502 20:46:53.913441 11578 raft_consensus.cc:686] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.913542 11578 consensus_queue.cc:237] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 6.23, Last appended by leader: 23, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.913700 9142 catalog_manager.cc:5484] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 7 to 8, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 8 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:53.917213 9136 catalog_manager.cc:5484] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 6 to 8, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 8 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:53.917769 9554 leader_election.cc:288] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:53.917937 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 6 index: 23 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.918032 9687 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:53.918323 9554 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.918419 9554 raft_consensus.cc:482] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.918493 9554 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.918849 9687 raft_consensus.cc:2454] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 8.
W20240502 20:46:53.920001 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59752: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.920830 9554 raft_consensus.cc:504] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.921000 9555 raft_consensus.cc:1226] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Rejecting Update request from peer 64136bd82a30486e9d56fc28da4bd448 for earlier term 6. Current term is 7. Ops: [6.41-6.41]
I20240502 20:46:53.921622 11839 consensus_queue.cc:1046] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: INVALID_TERM, Last received: 6.40, Next index: 41, Last known committed idx: 40, Time since last communication: 0.000s
I20240502 20:46:53.921898 11624 raft_consensus.cc:3032] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 6 LEADER]: Stepping down as leader of term 6
I20240502 20:46:53.921945 11624 raft_consensus.cc:729] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.922055 11624 consensus_queue.cc:260] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 41, Committed index: 41, Last appended: 6.41, Last appended by leader: 41, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:53.922425 11624 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Advancing to term 7
W20240502 20:46:53.926874 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.927923 9424 raft_consensus.cc:1261] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 6 index: 11. Preceding OpId from leader: term: 7 index: 11. (term mismatch)
I20240502 20:46:53.927982 9424 pending_rounds.cc:76] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc: Aborting all ops after (but not including) 10
I20240502 20:46:53.928006 9424 pending_rounds.cc:97] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc: Aborting uncommitted NO_OP operation due to leader change: 6.11
I20240502 20:46:53.928030 9424 raft_consensus.cc:2866] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 20:46:53.928373 11739 consensus_queue.cc:1035] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 20:46:53.928463 11739 consensus_queue.cc:1225] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Peer 88a19090795d47c2a1ea1dd979fb62bc log is divergent from this leader: its last log entry 3.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20240502 20:46:53.928975 9424 pending_rounds.cc:76] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc: Aborting all ops after (but not including) 9
I20240502 20:46:53.929018 9424 pending_rounds.cc:97] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc: Aborting uncommitted WRITE_OP operation due to leader change: 3.10
W20240502 20:46:53.929121 9424 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55548: Aborted: Op aborted by new leader
I20240502 20:46:53.929229 9424 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:55548 (ReqId={client: b18c9179ac354e28a48e16fdf1db50a7, seq_no=19, attempt_no=0}) took 1115 ms. Trace:
I20240502 20:46:53.929309 9424 rpcz_store.cc:276] 0502 20:46:52.814249 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 20:46:52.814286 (+ 37us) service_pool.cc:225] Handling call
0502 20:46:53.929220 (+1114934us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0502 20:46:52.814593 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 546ac55ccb0b4e91a17074a8ee8e99ef
0502 20:46:52.814656 (+ 63us) write_op.cc:424] Acquiring schema lock in shared mode
0502 20:46:52.814659 (+ 3us) write_op.cc:427] Acquired schema lock
0502 20:46:52.814661 (+ 2us) tablet.cc:660] Decoding operations
0502 20:46:52.814706 (+ 45us) write_op.cc:612] Acquiring the partition lock for write op
0502 20:46:52.814712 (+ 6us) write_op.cc:633] Partition lock acquired for write op
0502 20:46:52.814715 (+ 3us) tablet.cc:683] Acquiring locks for 1 operations
0502 20:46:52.814742 (+ 27us) tablet.cc:699] Row locks acquired
0502 20:46:52.814743 (+ 1us) write_op.cc:260] PREPARE: finished
0502 20:46:52.814769 (+ 26us) write_op.cc:270] Start()
0502 20:46:52.814795 (+ 26us) write_op.cc:276] Timestamp: P: 1714682812814751 usec, L: 0
0502 20:46:52.814796 (+ 1us) op_driver.cc:331] REPLICATION: starting
0502 20:46:52.814893 (+ 97us) log.cc:842] Serialized 354 byte log entry
0502 20:46:53.929078 (+1114185us) op_driver.cc:444] REPLICATION: finished
0502 20:46:53.929090 (+ 12us) op_driver.cc:372] HandleFailure(Aborted: Op aborted by new leader)
0502 20:46:53.929094 (+ 4us) write_op.cc:481] Releasing partition, row and schema locks
0502 20:46:53.929112 (+ 18us) write_op.cc:446] Released schema lock
0502 20:46:53.929116 (+ 4us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":1,"prepare.queue_time_us":37,"prepare.run_cpu_time_us":345,"prepare.run_wall_time_us":345,"replication_time_us":1114246}]]}
I20240502 20:46:53.933001 9555 raft_consensus.cc:1261] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 3 index: 9. Preceding OpId from leader: term: 7 index: 11. (index mismatch)
I20240502 20:46:53.933553 10895 consensus_queue.cc:1035] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 20:46:53.933739 9687 raft_consensus.cc:1261] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 20. Preceding OpId from leader: term: 8 index: 25. (index mismatch)
I20240502 20:46:53.934306 11578 consensus_queue.cc:1035] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 19, Time since last communication: 0.000s
W20240502 20:46:53.937930 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
W20240502 20:46:53.938148 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51626: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.939146 11797 mvcc.cc:205] Tried to move back new op lower bound from 7023340805868953600 to 7023340805791121408. Current Snapshot: MvccSnapshot[applied={T|T < 7023340805868953600}]
I20240502 20:46:53.939581 9286 raft_consensus.cc:1261] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 23. Preceding OpId from leader: term: 8 index: 25. (index mismatch)
I20240502 20:46:53.940279 11578 consensus_queue.cc:1035] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20240502 20:46:53.944968 9687 raft_consensus.cc:1261] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 7 index: 28. Preceding OpId from leader: term: 8 index: 29. (index mismatch)
I20240502 20:46:53.945040 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 40 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.945128 9286 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.945361 11578 consensus_queue.cc:1035] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20240502 20:46:53.945436 11578 consensus_queue.cc:1225] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Peer 64136bd82a30486e9d56fc28da4bd448 log is divergent from this leader: its last log entry 7.28 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 27
I20240502 20:46:53.946187 9286 raft_consensus.cc:2396] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 7 because replica has last-logged OpId of term: 6 index: 41, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 40.
I20240502 20:46:53.946545 9687 pending_rounds.cc:76] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448: Aborting all ops after (but not including) 27
I20240502 20:46:53.946585 9687 pending_rounds.cc:97] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448: Aborting uncommitted NO_OP operation due to leader change: 7.28
I20240502 20:46:53.946611 9687 raft_consensus.cc:2866] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 20:46:53.947363 9424 raft_consensus.cc:1261] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 27. Preceding OpId from leader: term: 8 index: 29. (index mismatch)
I20240502 20:46:53.947924 11578 consensus_queue.cc:1035] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
W20240502 20:46:53.952191 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59752: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
W20240502 20:46:53.963635 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.965521 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 40 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.965649 9687 raft_consensus.cc:2396] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 7 because replica has last-logged OpId of term: 6 index: 41, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 40.
W20240502 20:46:53.965687 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40118: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:53.965909 9492 leader_election.cc:302] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:53.966063 11532 raft_consensus.cc:2726] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20240502 20:46:53.966220 9554 leader_election.cc:288] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 7 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:53.967554 9555 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.967655 9555 raft_consensus.cc:482] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.967706 9555 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:53.968683 9555 raft_consensus.cc:504] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.969003 9555 leader_election.cc:288] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.969578 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.969699 9286 raft_consensus.cc:3032] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 LEADER]: Stepping down as leader of term 7
I20240502 20:46:53.969748 9286 raft_consensus.cc:729] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.969871 9286 consensus_queue.cc:260] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:53.970012 9286 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:53.970995 9286 raft_consensus.cc:2396] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20240502 20:46:53.971802 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.971915 9423 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:53.972954 9423 raft_consensus.cc:2396] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20240502 20:46:53.973762 9491 leader_election.cc:302] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc
W20240502 20:46:53.973951 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:53.974581 11577 raft_consensus.cc:2726] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20240502 20:46:53.978753 9554 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3315525f1849462fbf49a285f424ea51"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.978855 9554 raft_consensus.cc:482] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.978900 9554 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:53.979710 9554 raft_consensus.cc:504] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.980526 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3315525f1849462fbf49a285f424ea51" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 40 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.980631 9420 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:53.981499 9420 raft_consensus.cc:2454] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 8.
W20240502 20:46:53.981956 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51610: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.982424 9491 leader_election.cc:302] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:53.982580 11532 raft_consensus.cc:2781] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Leader election won for term 8
I20240502 20:46:53.982674 11532 raft_consensus.cc:686] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.982800 11532 consensus_queue.cc:237] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 7.40, Last appended by leader: 40, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
W20240502 20:46:53.983826 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51626: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:53.984776 9136 catalog_manager.cc:5484] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 7 to 8, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 8 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:53.986671 9554 leader_election.cc:288] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.987017 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3315525f1849462fbf49a285f424ea51" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 40 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.987121 9286 raft_consensus.cc:3032] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 LEADER]: Stepping down as leader of term 7
I20240502 20:46:53.987159 9286 raft_consensus.cc:729] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
W20240502 20:46:53.987257 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59728: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.989401 9554 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.989495 9554 raft_consensus.cc:482] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.989544 9554 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Advancing to term 8
W20240502 20:46:53.990658 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59752: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:53.990968 9554 raft_consensus.cc:504] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.991714 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 12 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:53.991824 9289 raft_consensus.cc:3032] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 LEADER]: Stepping down as leader of term 7
I20240502 20:46:53.991863 9289 raft_consensus.cc:729] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:53.991997 9289 consensus_queue.cc:260] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 7.12, Last appended by leader: 12, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.992165 9289 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:53.993052 9289 raft_consensus.cc:2454] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 8.
I20240502 20:46:53.993362 9490 leader_election.cc:302] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:53.987277 9286 consensus_queue.cc:260] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 7.40, Last appended by leader: 40, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.993901 11577 raft_consensus.cc:2781] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Leader election won for term 8
I20240502 20:46:53.993973 9286 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:53.993978 11577 raft_consensus.cc:686] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:53.994091 11577 consensus_queue.cc:237] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 7.12, Last appended by leader: 12, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:53.994848 9286 raft_consensus.cc:2454] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 8.
I20240502 20:46:53.995008 9554 leader_election.cc:288] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:53.995828 9554 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.995925 9554 raft_consensus.cc:482] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.996026 9554 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:53.996274 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 12 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:53.996376 9424 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:53.996877 9554 raft_consensus.cc:504] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.996961 9136 catalog_manager.cc:5484] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 7 to 8, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 8 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:53.997329 9424 raft_consensus.cc:2454] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 8.
I20240502 20:46:53.998138 9554 leader_election.cc:288] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 7 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:53.999255 9554 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "edebc185b24441da9504c78394378f4d"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:53.999294 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 13 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:53.999347 9554 raft_consensus.cc:482] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:53.999392 9554 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:53.999382 9688 raft_consensus.cc:3032] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 6 LEADER]: Stepping down as leader of term 6
I20240502 20:46:53.999419 9688 raft_consensus.cc:729] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:53.999545 9688 consensus_queue.cc:260] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 6.14, Last appended by leader: 14, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:53.999668 9688 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:54.000130 9554 raft_consensus.cc:504] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.000617 9688 raft_consensus.cc:2396] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 7 because replica has last-logged OpId of term: 6 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 13.
I20240502 20:46:54.000828 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edebc185b24441da9504c78394378f4d" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 5 index: 13 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.000922 9420 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.001783 9420 raft_consensus.cc:2454] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 8.
I20240502 20:46:54.001824 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 13 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.001922 9424 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:54.002606 9491 leader_election.cc:302] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:54.002758 11577 raft_consensus.cc:2781] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Leader election won for term 8
I20240502 20:46:54.002841 11577 raft_consensus.cc:686] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
W20240502 20:46:54.002887 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:54.002945 11577 consensus_queue.cc:237] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 5.13, Last appended by leader: 13, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.003618 9554 leader_election.cc:288] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:54.003818 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edebc185b24441da9504c78394378f4d" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 5 index: 13 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.003924 9688 raft_consensus.cc:3032] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 20:46:54.003970 9688 raft_consensus.cc:729] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.004099 9688 consensus_queue.cc:260] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 7.14, Last appended by leader: 14, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.004231 9688 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.005081 9688 raft_consensus.cc:2396] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 8 because replica has last-logged OpId of term: 7 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 13.
I20240502 20:46:54.005877 9554 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.005972 9554 raft_consensus.cc:482] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.006021 9554 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:54.007191 9424 raft_consensus.cc:2454] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 7.
I20240502 20:46:54.007478 9491 leader_election.cc:302] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:54.007711 11578 raft_consensus.cc:2781] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Leader election won for term 7
I20240502 20:46:54.007778 11578 raft_consensus.cc:686] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.007877 11578 consensus_queue.cc:237] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 5.13, Last appended by leader: 13, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.008003 9554 raft_consensus.cc:504] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.008328 9554 leader_election.cc:288] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 7 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.009083 9554 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.009179 9554 raft_consensus.cc:482] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.009227 9554 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:54.009945 9554 raft_consensus.cc:504] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
W20240502 20:46:54.010174 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40118: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:54.010231 9554 leader_election.cc:288] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 7 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.011066 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 11 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.011160 9686 raft_consensus.cc:3032] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 6 LEADER]: Stepping down as leader of term 6
I20240502 20:46:54.011196 9686 raft_consensus.cc:729] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.011305 9686 consensus_queue.cc:260] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 6.12, Last appended by leader: 12, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.011422 9686 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:54.012060 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 7 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.012207 9688 raft_consensus.cc:3032] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 6 LEADER]: Stepping down as leader of term 6
I20240502 20:46:54.012341 9688 raft_consensus.cc:729] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.012377 9686 raft_consensus.cc:2396] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 7 because replica has last-logged OpId of term: 6 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 11.
I20240502 20:46:54.012462 9688 consensus_queue.cc:260] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 6.13, Last appended by leader: 13, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.012845 9688 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:54.014209 9688 raft_consensus.cc:2396] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 7 because replica has last-logged OpId of term: 6 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 11.
W20240502 20:46:54.016237 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40188: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
W20240502 20:46:54.018983 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51610: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
W20240502 20:46:54.019908 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:54.020983 9554 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.021095 9554 raft_consensus.cc:482] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.021144 9554 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.021971 9554 raft_consensus.cc:504] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.022321 9554 leader_election.cc:288] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.022683 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 39 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.022801 9688 raft_consensus.cc:3032] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 20:46:54.022846 9688 raft_consensus.cc:729] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.022989 9688 consensus_queue.cc:260] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 39, Committed index: 39, Last appended: 7.39, Last appended by leader: 39, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.023154 9688 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.023137 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 39 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.023240 9289 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Advancing to term 8
W20240502 20:46:54.023538 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59834: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:54.024179 9289 raft_consensus.cc:2454] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 8.
I20240502 20:46:54.024358 9688 raft_consensus.cc:2454] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 8.
I20240502 20:46:54.024593 9490 leader_election.cc:302] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:54.024925 11532 raft_consensus.cc:2781] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Leader election won for term 8
I20240502 20:46:54.025014 11532 raft_consensus.cc:686] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.025115 11532 consensus_queue.cc:237] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 39, Committed index: 39, Last appended: 7.39, Last appended by leader: 39, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.027698 9136 catalog_manager.cc:5484] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 7 to 8, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 8 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:54.027858 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 7 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.027959 9286 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:54.028892 9286 raft_consensus.cc:2396] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 7 because replica has last-logged OpId of term: 5 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 11.
I20240502 20:46:54.029224 9490 leader_election.cc:302] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:54.029431 11578 raft_consensus.cc:2726] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20240502 20:46:54.029572 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 11 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.029664 9424 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:54.030563 9424 raft_consensus.cc:2454] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 7.
W20240502 20:46:54.031283 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59728: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
W20240502 20:46:54.033162 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40188: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:54.033489 9554 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "20698b2531674eefb9df3125a86653ba"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.033670 9554 raft_consensus.cc:482] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.033767 9554 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.036283 9554 raft_consensus.cc:504] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.037395 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20698b2531674eefb9df3125a86653ba" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 5 index: 16 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.037544 9286 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.038435 9286 raft_consensus.cc:2454] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 8.
I20240502 20:46:54.039832 9490 leader_election.cc:302] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
W20240502 20:46:54.040802 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59752: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:54.042659 9554 leader_election.cc:288] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
W20240502 20:46:54.043565 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40118: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:54.044103 9554 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.044202 11577 raft_consensus.cc:2781] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Leader election won for term 8
I20240502 20:46:54.044317 9554 raft_consensus.cc:482] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.044369 11577 raft_consensus.cc:686] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.044375 9554 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.044512 11577 consensus_queue.cc:237] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 5.16, Last appended by leader: 16, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.045238 9554 raft_consensus.cc:504] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.045537 9554 leader_election.cc:288] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.046103 9554 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.046212 9554 raft_consensus.cc:482] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.046277 9554 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.046834 9491 leader_election.cc:302] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:54.047233 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.047341 9687 raft_consensus.cc:3032] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 20:46:54.047379 9687 raft_consensus.cc:729] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
W20240502 20:46:54.047538 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51626: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:54.047503 9687 consensus_queue.cc:260] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.047626 9687 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.047868 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20698b2531674eefb9df3125a86653ba" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 5 index: 16 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.047960 9420 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.048130 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.048231 9424 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.048584 9687 raft_consensus.cc:2396] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20240502 20:46:54.048864 9420 raft_consensus.cc:2396] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 8 because replica has last-logged OpId of term: 6 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 16.
I20240502 20:46:54.049103 9424 raft_consensus.cc:2454] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 8.
I20240502 20:46:54.049144 11578 raft_consensus.cc:2781] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Leader election won for term 7
I20240502 20:46:54.049222 11578 raft_consensus.cc:686] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.049322 11578 consensus_queue.cc:237] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 4.11, Last appended by leader: 11, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.049502 9554 raft_consensus.cc:504] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.049919 9554 leader_election.cc:288] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:54.050345 9491 leader_election.cc:302] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:54.050676 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 5 index: 13 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.050835 9424 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.050832 11578 raft_consensus.cc:2781] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Leader election won for term 8
I20240502 20:46:54.051203 9136 catalog_manager.cc:5484] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 6 to 7, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 7 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:54.051440 9136 catalog_manager.cc:5484] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 6 to 8, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 8 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:54.051632 9136 catalog_manager.cc:5484] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 7 to 8, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 8 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:54.051903 9424 raft_consensus.cc:2454] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 8.
I20240502 20:46:54.052237 9554 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.052399 9554 raft_consensus.cc:482] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.052495 9554 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.052577 11578 raft_consensus.cc:686] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.052713 11578 consensus_queue.cc:237] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
W20240502 20:46:54.054315 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51610: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:54.053467 9554 raft_consensus.cc:504] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.055325 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 5 index: 13 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.055429 9289 raft_consensus.cc:3032] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 LEADER]: Stepping down as leader of term 7
I20240502 20:46:54.055464 9289 raft_consensus.cc:729] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.055604 9289 consensus_queue.cc:260] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 7.14, Last appended by leader: 14, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.055733 9289 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.056660 9289 raft_consensus.cc:2396] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 8 because replica has last-logged OpId of term: 7 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 13.
W20240502 20:46:54.057439 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
W20240502 20:46:54.058419 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59834: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:54.056306 9491 leader_election.cc:302] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:54.060671 9136 catalog_manager.cc:5484] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 6 to 7, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 7 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:54.060863 9136 catalog_manager.cc:5484] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 7 to 8, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 8 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:54.060945 11578 raft_consensus.cc:2781] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Leader election won for term 8
I20240502 20:46:54.061254 11578 raft_consensus.cc:686] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.061383 11578 consensus_queue.cc:237] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 5.13, Last appended by leader: 13, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.063611 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 5 index: 16 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.063711 9289 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.064371 9289 raft_consensus.cc:2396] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 8 because replica has last-logged OpId of term: 6 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 16.
I20240502 20:46:54.064311 9136 catalog_manager.cc:5484] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 7 to 8, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 8 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
W20240502 20:46:54.068053 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59728: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
W20240502 20:46:54.070408 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40188: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:54.071493 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 5 index: 16 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.071606 9686 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.072573 9686 raft_consensus.cc:2454] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 8.
I20240502 20:46:54.073324 9492 leader_election.cc:302] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters: 4b3cb6ad00954dc5a1270dc392870b4f
I20240502 20:46:54.073596 11532 raft_consensus.cc:2781] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Leader election won for term 8
I20240502 20:46:54.073670 11532 raft_consensus.cc:686] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.073772 11532 consensus_queue.cc:237] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 5.16, Last appended by leader: 16, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.075640 9136 catalog_manager.cc:5484] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 6 to 8, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 8 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
W20240502 20:46:54.076114 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:54.077204 9554 leader_election.cc:288] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.078569 9555 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.078687 9555 raft_consensus.cc:482] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.078747 9555 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Advancing to term 8
W20240502 20:46:54.079308 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51626: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:54.079679 9555 raft_consensus.cc:504] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.081192 9555 leader_election.cc:288] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.081691 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 18 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.081796 9420 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.082535 9420 raft_consensus.cc:2454] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 8.
W20240502 20:46:54.082573 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40118: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:54.082803 9491 leader_election.cc:302] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:54.083060 11578 raft_consensus.cc:2781] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Leader election won for term 8
I20240502 20:46:54.083129 11578 raft_consensus.cc:686] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.083227 11578 consensus_queue.cc:237] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 7.18, Last appended by leader: 18, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.083313 9555 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.083403 9555 raft_consensus.cc:482] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.083449 9555 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.084226 9555 raft_consensus.cc:504] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.084558 9555 leader_election.cc:288] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:54.085036 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 18 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.085013 9136 catalog_manager.cc:5484] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 7 to 8, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 8 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:54.085124 9688 raft_consensus.cc:3032] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 20:46:54.085157 9688 raft_consensus.cc:729] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.085287 9688 consensus_queue.cc:260] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 7.18, Last appended by leader: 18, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.085451 9688 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.085516 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.085606 9687 raft_consensus.cc:3032] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 20:46:54.085649 9687 raft_consensus.cc:729] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.085764 9687 consensus_queue.cc:260] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.085872 9687 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.086236 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.086330 9688 raft_consensus.cc:2454] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 8.
I20240502 20:46:54.086782 9687 raft_consensus.cc:2396] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20240502 20:46:54.087086 9424 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.087647 9555 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.087755 9555 raft_consensus.cc:482] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.087826 9555 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:54.088188 9424 raft_consensus.cc:2454] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 8.
I20240502 20:46:54.088570 9555 raft_consensus.cc:504] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.088898 9555 leader_election.cc:288] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 7 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.089279 9491 leader_election.cc:302] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:54.089525 11578 raft_consensus.cc:2781] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Leader election won for term 8
I20240502 20:46:54.089607 11578 raft_consensus.cc:686] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.089704 11578 consensus_queue.cc:237] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.091008 9555 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.091117 9555 raft_consensus.cc:482] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.091177 9555 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:54.091241 9136 catalog_manager.cc:5484] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 7 to 8, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 8 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:54.091929 9555 raft_consensus.cc:504] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.092257 9555 leader_election.cc:288] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 7 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.092710 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 7 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.092833 9289 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:54.093420 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 7 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.093518 9686 raft_consensus.cc:3032] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 6 LEADER]: Stepping down as leader of term 6
I20240502 20:46:54.093556 9686 raft_consensus.cc:729] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.093667 9686 consensus_queue.cc:260] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 6.17, Last appended by leader: 17, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.093750 9289 raft_consensus.cc:2396] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 7 because replica has last-logged OpId of term: 5 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 15.
I20240502 20:46:54.093803 9686 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:54.094600 9555 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
W20240502 20:46:54.094671 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59834: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:54.094722 9686 raft_consensus.cc:2396] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 7 because replica has last-logged OpId of term: 6 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 15.
I20240502 20:46:54.094755 9555 raft_consensus.cc:482] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.094940 9555 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 6 FOLLOWER]: Advancing to term 7
W20240502 20:46:54.096153 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51610: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
W20240502 20:46:54.097703 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59752: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:54.098009 9555 raft_consensus.cc:504] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.098541 9555 leader_election.cc:288] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 7 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.098524 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 27 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.098563 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 11 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.098624 9420 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:54.098646 9286 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:54.098687 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 27 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.098770 9289 raft_consensus.cc:3032] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Stepping down as leader of term 6
I20240502 20:46:54.098804 9289 raft_consensus.cc:729] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.098928 9289 consensus_queue.cc:260] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 6.27, Last appended by leader: 27, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.099067 9289 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:54.099359 9286 raft_consensus.cc:2454] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 7.
I20240502 20:46:54.099550 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 11 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.099655 9424 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:54.099638 9490 leader_election.cc:302] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:54.099561 9420 raft_consensus.cc:2454] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 7.
I20240502 20:46:54.099799 11577 raft_consensus.cc:2781] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Leader election won for term 7
I20240502 20:46:54.099856 9289 raft_consensus.cc:2454] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 7.
I20240502 20:46:54.099869 11577 raft_consensus.cc:686] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.099967 11577 consensus_queue.cc:237] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 5.11, Last appended by leader: 11, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.100045 9491 leader_election.cc:302] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:54.100229 10554 raft_consensus.cc:2781] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Leader election won for term 7
I20240502 20:46:54.100301 10554 raft_consensus.cc:686] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.100394 10554 consensus_queue.cc:237] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 6.27, Last appended by leader: 27, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.100440 9424 raft_consensus.cc:2454] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 7.
I20240502 20:46:54.098281 9492 leader_election.cc:302] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:54.101229 11532 raft_consensus.cc:2726] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20240502 20:46:54.101716 9142 catalog_manager.cc:5484] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
W20240502 20:46:54.103250 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59778: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:54.104900 9424 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:54.105008 9424 raft_consensus.cc:482] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.105070 9424 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.105356 9554 raft_consensus.cc:1226] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 LEADER]: Rejecting Update request from peer 4b3cb6ad00954dc5a1270dc392870b4f for earlier term 6. Current term is 7. Ops: []
I20240502 20:46:54.105638 9136 catalog_manager.cc:5484] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 6 to 7, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 7 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:54.105819 9424 raft_consensus.cc:504] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.106164 9424 leader_election.cc:288] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 9 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.107262 9424 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:54.107362 9424 raft_consensus.cc:482] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.107411 9424 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:54.107527 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.107590 9554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.107618 9289 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.107674 9554 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.108201 9424 raft_consensus.cc:504] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.108454 9289 raft_consensus.cc:2396] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20240502 20:46:54.108554 9554 raft_consensus.cc:2454] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 9.
I20240502 20:46:54.108695 9424 leader_election.cc:288] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 7 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.108769 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.108834 9357 leader_election.cc:302] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:54.108868 9686 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:54.109045 11710 raft_consensus.cc:2781] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Leader election won for term 9
I20240502 20:46:54.109128 11710 raft_consensus.cc:686] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 9 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.109238 11710 consensus_queue.cc:237] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.109405 9424 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:54.109499 9424 raft_consensus.cc:482] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.109546 9424 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.109699 9686 raft_consensus.cc:2454] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 7.
I20240502 20:46:54.109984 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.110075 9286 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:54.110239 9424 raft_consensus.cc:504] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
W20240502 20:46:54.110389 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40188: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:54.110589 9424 leader_election.cc:288] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 9 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:54.110981 9286 raft_consensus.cc:2396] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20240502 20:46:54.111359 9357 leader_election.cc:302] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 4b3cb6ad00954dc5a1270dc392870b4f
I20240502 20:46:54.111392 9286 raft_consensus.cc:1261] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 27. Preceding OpId from leader: term: 7 index: 29. (index mismatch)
I20240502 20:46:54.111474 9554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 9 candidate_status { last_received { term: 8 index: 37 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.111560 9554 raft_consensus.cc:3032] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Stepping down as leader of term 8
I20240502 20:46:54.111619 9554 raft_consensus.cc:729] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.111690 11712 raft_consensus.cc:2781] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Leader election won for term 7
I20240502 20:46:54.111682 10554 consensus_queue.cc:1035] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20240502 20:46:54.111765 11712 raft_consensus.cc:686] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.111737 9554 consensus_queue.cc:260] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 37, Committed index: 37, Last appended: 8.37, Last appended by leader: 37, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.111883 9554 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.111861 11712 consensus_queue.cc:237] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
W20240502 20:46:54.112258 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59728: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:54.112726 9554 raft_consensus.cc:2454] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 9.
I20240502 20:46:54.113106 9357 leader_election.cc:302] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:54.113252 11710 raft_consensus.cc:2781] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Leader election won for term 9
I20240502 20:46:54.113337 11710 raft_consensus.cc:686] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.113437 11710 consensus_queue.cc:237] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 37, Committed index: 37, Last appended: 8.37, Last appended by leader: 37, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.113457 9136 catalog_manager.cc:5484] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 7 to 9, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 9 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:54.113555 9424 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "edebc185b24441da9504c78394378f4d"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:54.113688 9424 raft_consensus.cc:482] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.113740 9424 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.114224 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 9 candidate_status { last_received { term: 8 index: 37 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.114324 9686 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.114429 9424 raft_consensus.cc:504] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.114652 9420 raft_consensus.cc:1261] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 27. Preceding OpId from leader: term: 7 index: 29. (index mismatch)
I20240502 20:46:54.115056 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edebc185b24441da9504c78394378f4d" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 9 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.115159 9687 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.115579 9424 leader_election.cc:288] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 9 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:54.115700 9554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edebc185b24441da9504c78394378f4d" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 9 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.115945 9554 raft_consensus.cc:3032] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Stepping down as leader of term 8
I20240502 20:46:54.116035 9554 raft_consensus.cc:729] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.116125 9424 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:54.116230 9424 raft_consensus.cc:482] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.116288 9424 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.116266 9554 consensus_queue.cc:260] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 8.14, Last appended by leader: 14, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.116546 9554 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.117103 9424 raft_consensus.cc:504] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.117290 11532 consensus_queue.cc:1035] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20240502 20:46:54.117503 9687 raft_consensus.cc:2396] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 9 because replica has last-logged OpId of term: 7 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 12.
I20240502 20:46:54.118016 9424 leader_election.cc:288] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 9 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:54.118243 9686 raft_consensus.cc:2454] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 9.
I20240502 20:46:54.118257 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.118207 9554 raft_consensus.cc:2396] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 9 because replica has last-logged OpId of term: 8 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 12.
I20240502 20:46:54.118359 9289 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.118829 9357 leader_election.cc:302] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448
W20240502 20:46:54.118875 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:54.119041 11712 raft_consensus.cc:2726] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20240502 20:46:54.120075 9289 raft_consensus.cc:2396] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20240502 20:46:54.120157 9142 catalog_manager.cc:5484] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 8 to 9, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 9 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:54.120355 9142 catalog_manager.cc:5484] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 5 to 7, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 7 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:54.120890 9420 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3315525f1849462fbf49a285f424ea51"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:54.120988 9420 raft_consensus.cc:482] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.121035 9420 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.121827 9420 raft_consensus.cc:504] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.122375 9554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3315525f1849462fbf49a285f424ea51" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 9 candidate_status { last_received { term: 7 index: 40 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.122498 9554 raft_consensus.cc:3032] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Stepping down as leader of term 8
I20240502 20:46:54.122540 9554 raft_consensus.cc:729] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.122673 9554 consensus_queue.cc:260] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 8.41, Last appended by leader: 41, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.122808 9554 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.123574 9554 raft_consensus.cc:2396] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 9 because replica has last-logged OpId of term: 8 index: 41, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 40.
W20240502 20:46:54.123553 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51626: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
W20240502 20:46:54.124097 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51672: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:54.126251 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.127539 9420 leader_election.cc:288] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 9 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.127936 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3315525f1849462fbf49a285f424ea51" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 9 candidate_status { last_received { term: 7 index: 40 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.128033 9289 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.128525 9420 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
W20240502 20:46:54.128585 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40118: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:54.128638 9420 raft_consensus.cc:482] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.128698 9420 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.128726 9289 raft_consensus.cc:2454] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 9.
I20240502 20:46:54.128989 9355 leader_election.cc:302] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e
I20240502 20:46:54.129189 11710 raft_consensus.cc:2781] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Leader election won for term 9
I20240502 20:46:54.129263 11710 raft_consensus.cc:686] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.129380 11710 consensus_queue.cc:237] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 7.40, Last appended by leader: 40, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.129500 9555 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.130296 9555 raft_consensus.cc:2454] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 9.
I20240502 20:46:54.131034 9138 catalog_manager.cc:5484] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 8 to 9, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 9 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:54.132751 9686 raft_consensus.cc:1261] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 8 index: 37. Preceding OpId from leader: term: 9 index: 39. (index mismatch)
I20240502 20:46:54.133239 9555 raft_consensus.cc:1261] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 8 index: 37. Preceding OpId from leader: term: 9 index: 39. (index mismatch)
I20240502 20:46:54.133252 9357 leader_election.cc:302] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 4b3cb6ad00954dc5a1270dc392870b4f
I20240502 20:46:54.133409 11710 raft_consensus.cc:2781] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Leader election won for term 9
I20240502 20:46:54.133498 11712 consensus_queue.cc:1035] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 38, Last known committed idx: 37, Time since last communication: 0.000s
I20240502 20:46:54.133561 11710 raft_consensus.cc:686] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 9 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.133671 11710 consensus_queue.cc:237] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.135280 11712 consensus_queue.cc:1035] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 38, Last known committed idx: 37, Time since last communication: 0.000s
I20240502 20:46:54.136010 9420 raft_consensus.cc:504] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.136550 9420 leader_election.cc:288] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 9 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.137293 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 9 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.137393 9686 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.137670 9420 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:54.137825 9420 raft_consensus.cc:482] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.137933 9420 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.138232 9686 raft_consensus.cc:2396] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20240502 20:46:54.139132 9420 raft_consensus.cc:504] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
W20240502 20:46:54.139290 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:54.139647 9420 leader_election.cc:288] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 9 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.140058 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 9 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.140156 9286 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.140386 9420 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:54.140535 9420 raft_consensus.cc:482] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
W20240502 20:46:54.140741 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59834: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:54.140743 9420 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.141013 9286 raft_consensus.cc:2396] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 9 because replica has last-logged OpId of term: 7 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 12.
I20240502 20:46:54.141482 9420 raft_consensus.cc:504] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.143276 9139 catalog_manager.cc:5484] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 7 to 9, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 9 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:54.145665 9420 leader_election.cc:288] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 8 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.145686 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 8 candidate_status { last_received { term: 5 index: 13 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.145787 9686 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Advancing to term 8
W20240502 20:46:54.145890 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51610: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:54.146607 9686 raft_consensus.cc:2396] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 8 because replica has last-logged OpId of term: 6 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 13.
I20240502 20:46:54.147145 9420 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fe22224142984d69a2def286584b9aaa"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:54.147254 9420 raft_consensus.cc:482] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.147316 9420 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.147405 9554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 9 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.147500 9554 raft_consensus.cc:3032] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Stepping down as leader of term 8
I20240502 20:46:54.147580 9554 raft_consensus.cc:729] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.147622 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 8 candidate_status { last_received { term: 5 index: 13 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.147704 9553 raft_consensus.cc:3032] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 LEADER]: Stepping down as leader of term 7
I20240502 20:46:54.147765 9553 raft_consensus.cc:729] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.147707 9554 consensus_queue.cc:260] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.147493 9550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 9 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.147902 9554 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.147984 9550 raft_consensus.cc:3032] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Stepping down as leader of term 8
I20240502 20:46:54.148033 9550 raft_consensus.cc:729] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.148154 9550 consensus_queue.cc:260] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 8.14, Last appended by leader: 14, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.148288 9550 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.148679 9554 raft_consensus.cc:2396] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20240502 20:46:54.148931 9357 leader_election.cc:302] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:54.149170 9550 raft_consensus.cc:2396] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 9 because replica has last-logged OpId of term: 8 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 12.
I20240502 20:46:54.149268 11712 raft_consensus.cc:2726] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20240502 20:46:54.147900 9553 consensus_queue.cc:260] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 7.14, Last appended by leader: 14, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.149470 9553 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.150138 9357 leader_election.cc:302] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f
I20240502 20:46:54.150303 9553 raft_consensus.cc:2396] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 8 because replica has last-logged OpId of term: 7 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 13.
I20240502 20:46:54.150521 9423 raft_consensus.cc:1261] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20240502 20:46:54.150615 9357 leader_election.cc:302] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:54.150805 11712 raft_consensus.cc:2726] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20240502 20:46:54.150892 11712 raft_consensus.cc:2726] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20240502 20:46:54.150904 11624 consensus_queue.cc:1035] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
W20240502 20:46:54.155076 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40188: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:54.155310 9286 raft_consensus.cc:1261] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20240502 20:46:54.155361 9286 pending_rounds.cc:76] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f: Aborting all ops after (but not including) 7
I20240502 20:46:54.155383 9286 pending_rounds.cc:97] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20240502 20:46:54.155409 9286 raft_consensus.cc:2866] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 20:46:54.155841 11839 consensus_queue.cc:1035] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 20:46:54.155931 11839 consensus_queue.cc:1225] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Peer 4b3cb6ad00954dc5a1270dc392870b4f log is divergent from this leader: its last log entry 3.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20240502 20:46:54.156057 9420 raft_consensus.cc:504] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.156364 9420 leader_election.cc:288] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 8 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.156461 9289 pending_rounds.cc:76] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f: Aborting all ops after (but not including) 6
I20240502 20:46:54.156499 9289 pending_rounds.cc:97] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f: Aborting uncommitted NO_OP operation due to leader change: 3.7
I20240502 20:46:54.156525 9289 raft_consensus.cc:2866] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 20:46:54.156913 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe22224142984d69a2def286584b9aaa" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 8 candidate_status { last_received { term: 5 index: 20 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.157006 9287 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.157886 9287 raft_consensus.cc:2396] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 8 because replica has last-logged OpId of term: 6 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 20.
I20240502 20:46:54.158215 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe22224142984d69a2def286584b9aaa" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 8 candidate_status { last_received { term: 5 index: 20 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.158319 9686 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.159256 9686 raft_consensus.cc:2454] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 8.
I20240502 20:46:54.159500 9357 leader_election.cc:302] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:54.159667 11710 raft_consensus.cc:2781] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Leader election won for term 8
I20240502 20:46:54.159741 11710 raft_consensus.cc:686] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 8 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.159853 11710 consensus_queue.cc:237] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 5.20, Last appended by leader: 20, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
W20240502 20:46:54.161473 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59752: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
W20240502 20:46:54.162350 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59728: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:54.162449 9142 catalog_manager.cc:5484] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 6 to 8, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 8 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:54.163854 9423 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:54.163949 9423 raft_consensus.cc:482] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.163997 9423 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.164839 9423 raft_consensus.cc:504] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.165146 9423 leader_election.cc:288] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 8 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.166013 9423 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:54.166111 9423 raft_consensus.cc:482] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.166160 9423 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.167268 9423 raft_consensus.cc:504] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.167518 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 8 candidate_status { last_received { term: 2 index: 10 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.167613 9686 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.167675 9423 leader_election.cc:288] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 8 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.167958 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 8 candidate_status { last_received { term: 5 index: 17 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.168056 9287 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.168226 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 8 candidate_status { last_received { term: 5 index: 17 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.168310 9687 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.168665 9686 raft_consensus.cc:2396] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 8 because replica has last-logged OpId of term: 6 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 10.
I20240502 20:46:54.168929 9287 raft_consensus.cc:2396] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 8 because replica has last-logged OpId of term: 6 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 17.
I20240502 20:46:54.169154 9687 raft_consensus.cc:2454] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 8.
I20240502 20:46:54.170410 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 8 candidate_status { last_received { term: 2 index: 10 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
W20240502 20:46:54.170512 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51626: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:54.170526 9553 raft_consensus.cc:3032] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 LEADER]: Stepping down as leader of term 7
I20240502 20:46:54.170712 9357 leader_election.cc:302] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 4b3cb6ad00954dc5a1270dc392870b4f
I20240502 20:46:54.170769 9553 raft_consensus.cc:729] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.170869 11889 raft_consensus.cc:2781] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Leader election won for term 8
I20240502 20:46:54.170964 11889 raft_consensus.cc:686] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.170909 9553 consensus_queue.cc:260] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 7.12, Last appended by leader: 12, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.171061 9553 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.171072 11889 consensus_queue.cc:237] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 5.17, Last appended by leader: 17, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.171201 9423 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:54.171294 9423 raft_consensus.cc:482] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.171343 9423 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.171926 9553 raft_consensus.cc:2396] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 8 because replica has last-logged OpId of term: 7 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 10.
I20240502 20:46:54.172107 9423 raft_consensus.cc:504] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.172386 9423 leader_election.cc:288] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 9 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.172976 9357 leader_election.cc:302] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:54.173117 11712 raft_consensus.cc:2726] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20240502 20:46:54.173100 9142 catalog_manager.cc:5484] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 6 to 8, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 8 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:54.173348 9423 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:54.173437 9423 raft_consensus.cc:482] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.173485 9423 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.174649 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 9 candidate_status { last_received { term: 7 index: 12 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.174779 9286 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.175154 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 9 candidate_status { last_received { term: 7 index: 12 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.175253 9553 raft_consensus.cc:3032] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Stepping down as leader of term 8
I20240502 20:46:54.175297 9553 raft_consensus.cc:729] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.175422 9553 consensus_queue.cc:260] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 8.13, Last appended by leader: 13, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.175544 9553 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.176270 9286 raft_consensus.cc:2454] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 9.
I20240502 20:46:54.176307 9553 raft_consensus.cc:2396] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 9 because replica has last-logged OpId of term: 8 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 12.
I20240502 20:46:54.177022 9423 raft_consensus.cc:504] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.177325 9423 leader_election.cc:288] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 8 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.177541 9355 leader_election.cc:302] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e
I20240502 20:46:54.177814 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.177914 9687 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.178006 11712 raft_consensus.cc:2781] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Leader election won for term 9
I20240502 20:46:54.178094 11712 raft_consensus.cc:686] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 9 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.178197 11712 consensus_queue.cc:237] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 7.12, Last appended by leader: 12, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.178787 9687 raft_consensus.cc:2396] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20240502 20:46:54.179530 9424 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:54.179631 9424 raft_consensus.cc:482] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.179682 9424 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.180585 9424 raft_consensus.cc:504] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.181232 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.181336 9687 raft_consensus.cc:3032] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 20:46:54.181386 9687 raft_consensus.cc:729] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.181519 9687 consensus_queue.cc:260] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.181684 9687 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.182482 9687 raft_consensus.cc:2454] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 8.
I20240502 20:46:54.182834 9357 leader_election.cc:302] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:54.183434 11712 raft_consensus.cc:2781] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Leader election won for term 8
I20240502 20:46:54.183507 11712 raft_consensus.cc:686] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 8 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.183607 11712 consensus_queue.cc:237] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
W20240502 20:46:54.184126 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40118: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:54.185427 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.185516 9553 raft_consensus.cc:3032] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 LEADER]: Stepping down as leader of term 7
I20240502 20:46:54.185550 9553 raft_consensus.cc:729] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.185685 9553 consensus_queue.cc:260] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.185822 9553 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Advancing to term 8
W20240502 20:46:54.186228 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59834: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:54.186631 9553 raft_consensus.cc:2396] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20240502 20:46:54.186822 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:54.186962 9424 leader_election.cc:288] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 8 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.187975 9357 leader_election.cc:302] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:54.188102 11889 raft_consensus.cc:2726] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20240502 20:46:54.188261 9424 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:54.188357 9424 raft_consensus.cc:482] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.188405 9424 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.179616 9142 catalog_manager.cc:5484] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 8 to 9, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 9 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:54.191568 9424 raft_consensus.cc:504] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.192044 9420 raft_consensus.cc:1226] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Rejecting Update request from peer 190c1d3968d949bbbaa27a51a2b91d8e for earlier term 7. Current term is 8. Ops: [7.33-7.33]
I20240502 20:46:54.192430 11532 consensus_queue.cc:1046] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: INVALID_TERM, Last received: 7.32, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20240502 20:46:54.192596 11532 raft_consensus.cc:3032] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 LEADER]: Stepping down as leader of term 7
I20240502 20:46:54.192639 11532 raft_consensus.cc:729] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.192747 11532 consensus_queue.cc:260] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 7.33, Last appended by leader: 33, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.192799 9424 leader_election.cc:288] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 8 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.192883 11532 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.194104 9424 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:54.194223 9424 raft_consensus.cc:482] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.194269 9424 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.195084 9424 raft_consensus.cc:504] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.195536 9139 catalog_manager.cc:5484] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 7 to 8, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 8 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:54.195824 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 8 candidate_status { last_received { term: 5 index: 10 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.195916 9555 raft_consensus.cc:3032] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 LEADER]: Stepping down as leader of term 7
I20240502 20:46:54.195951 9555 raft_consensus.cc:729] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.196061 9555 consensus_queue.cc:260] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 7.12, Last appended by leader: 12, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.196184 9555 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.196632 9536 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 20:46:54.197069 9555 raft_consensus.cc:2396] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 8 because replica has last-logged OpId of term: 7 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 10.
I20240502 20:46:54.197443 9424 leader_election.cc:288] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 8 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:54.201843 11031 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:54.201833 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.201929 9287 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.202625 9667 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 20:46:54.202806 9269 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:54.203523 9287 raft_consensus.cc:2454] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 8.
I20240502 20:46:54.204030 9424 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d92bfee8fabd46a28ffcb7e319879406"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:54.204133 9424 raft_consensus.cc:482] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.204180 9424 raft_consensus.cc:3037] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:54.192642 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 8 candidate_status { last_received { term: 7 index: 32 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.205186 9553 raft_consensus.cc:2396] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 8 because replica has last-logged OpId of term: 7 index: 33, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 32.
I20240502 20:46:54.205627 9424 raft_consensus.cc:504] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.206228 9424 leader_election.cc:288] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 7 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
W20240502 20:46:54.207149 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51610: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:54.209630 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d92bfee8fabd46a28ffcb7e319879406" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 7 candidate_status { last_received { term: 6 index: 20 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.209736 9687 raft_consensus.cc:3032] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 6 LEADER]: Stepping down as leader of term 6
I20240502 20:46:54.209782 9687 raft_consensus.cc:729] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.209915 9687 consensus_queue.cc:260] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 6.20, Last appended by leader: 20, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.210057 9687 raft_consensus.cc:3037] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 6 FOLLOWER]: Advancing to term 7
W20240502 20:46:54.210839 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40188: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:54.211192 9420 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:54.211301 9420 raft_consensus.cc:482] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.211347 9420 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.211575 9687 raft_consensus.cc:2454] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 7.
I20240502 20:46:54.212241 9420 raft_consensus.cc:504] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.212546 9420 leader_election.cc:288] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 8 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.212775 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 8 candidate_status { last_received { term: 7 index: 32 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.212863 9287 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.218287 9287 raft_consensus.cc:2396] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 8 because replica has last-logged OpId of term: 7 index: 33, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 32.
I20240502 20:46:54.218524 11164 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:54.218636 9357 leader_election.cc:302] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:54.218793 11712 raft_consensus.cc:2781] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Leader election won for term 7
I20240502 20:46:54.218858 11712 raft_consensus.cc:686] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.218942 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 8 candidate_status { last_received { term: 5 index: 13 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.218955 11712 consensus_queue.cc:237] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 6.20, Last appended by leader: 20, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.219038 9686 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.219486 9420 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "20698b2531674eefb9df3125a86653ba"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:54.219591 9420 raft_consensus.cc:482] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.219651 9420 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.219878 9686 raft_consensus.cc:2454] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 8.
I20240502 20:46:54.220433 9420 raft_consensus.cc:504] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.220604 9139 catalog_manager.cc:5484] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 6 to 7, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 7 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:54.220757 9420 leader_election.cc:288] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 9 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:54.221439 9420 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:54.221535 9420 raft_consensus.cc:482] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.221583 9420 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.222337 9420 raft_consensus.cc:504] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.222618 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20698b2531674eefb9df3125a86653ba" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 9 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.222723 9553 raft_consensus.cc:3032] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Stepping down as leader of term 8
I20240502 20:46:54.222771 9553 raft_consensus.cc:729] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.222891 9553 consensus_queue.cc:260] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 8.17, Last appended by leader: 17, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.223006 9553 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.223467 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 9 candidate_status { last_received { term: 7 index: 18 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.223572 9555 raft_consensus.cc:3032] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Stepping down as leader of term 8
I20240502 20:46:54.223614 9555 raft_consensus.cc:729] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.223737 9555 consensus_queue.cc:260] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 8.19, Last appended by leader: 19, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.223837 9553 raft_consensus.cc:2396] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 9 because replica has last-logged OpId of term: 8 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 16.
I20240502 20:46:54.223870 9555 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.223982 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 8 candidate_status { last_received { term: 5 index: 10 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.224066 9286 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.224720 9555 raft_consensus.cc:2396] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 9 because replica has last-logged OpId of term: 8 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 18.
I20240502 20:46:54.224895 9286 raft_consensus.cc:2454] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 8.
I20240502 20:46:54.225116 9420 leader_election.cc:288] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 9 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:54.225317 9357 leader_election.cc:302] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:54.225520 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 9 candidate_status { last_received { term: 7 index: 18 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.225545 11889 raft_consensus.cc:2781] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Leader election won for term 8
I20240502 20:46:54.225577 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d92bfee8fabd46a28ffcb7e319879406" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 7 candidate_status { last_received { term: 6 index: 20 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.225618 9686 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.225623 11889 raft_consensus.cc:686] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 8 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.225677 9286 raft_consensus.cc:3037] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 6 FOLLOWER]: Advancing to term 7
I20240502 20:46:54.225744 11889 consensus_queue.cc:237] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 5.13, Last appended by leader: 13, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.226403 9686 raft_consensus.cc:2454] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 9.
I20240502 20:46:54.226496 9286 raft_consensus.cc:2454] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 7.
I20240502 20:46:54.226716 9424 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:54.226848 9424 raft_consensus.cc:482] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.226866 9687 raft_consensus.cc:1261] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 4 index: 16. Preceding OpId from leader: term: 8 index: 19. (index mismatch)
I20240502 20:46:54.226902 9424 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.227666 9424 raft_consensus.cc:504] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.227870 9139 catalog_manager.cc:5484] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 6 to 8, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 8 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:54.227967 9424 leader_election.cc:288] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 9 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:54.226737 9357 leader_election.cc:302] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e
I20240502 20:46:54.228766 11712 raft_consensus.cc:2781] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Leader election won for term 9
I20240502 20:46:54.228768 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 9 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.228854 9687 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.228852 11712 raft_consensus.cc:686] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.228961 11712 consensus_queue.cc:237] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 7.18, Last appended by leader: 18, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.229415 11712 consensus_queue.cc:1035] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20240502 20:46:54.229684 9687 raft_consensus.cc:2396] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 7.
I20240502 20:46:54.230062 9424 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
W20240502 20:46:54.230100 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59728: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:54.230156 9424 raft_consensus.cc:482] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.230203 9424 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.231003 9424 raft_consensus.cc:504] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.231314 9424 leader_election.cc:288] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 8 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.231262 9139 catalog_manager.cc:5484] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 8 to 9, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 9 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:54.231825 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 8 candidate_status { last_received { term: 5 index: 11 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.231913 9686 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Advancing to term 8
W20240502 20:46:54.232180 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51626: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:54.232683 9686 raft_consensus.cc:2454] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 8.
I20240502 20:46:54.232949 9357 leader_election.cc:302] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:54.233155 11889 raft_consensus.cc:2781] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Leader election won for term 8
I20240502 20:46:54.233232 11889 raft_consensus.cc:686] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.233213 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 9 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.233304 9555 raft_consensus.cc:3032] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Stepping down as leader of term 8
I20240502 20:46:54.233342 9555 raft_consensus.cc:729] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.233341 11889 consensus_queue.cc:237] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 5.11, Last appended by leader: 11, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.233460 9555 consensus_queue.cc:260] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.233584 9555 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.234293 9286 raft_consensus.cc:1261] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 6 index: 17. Preceding OpId from leader: term: 8 index: 19. (index mismatch)
I20240502 20:46:54.234407 9555 raft_consensus.cc:2396] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 7.
I20240502 20:46:54.234898 9139 catalog_manager.cc:5484] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 6 to 8, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 8 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:54.235632 9357 leader_election.cc:302] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:54.235663 11436 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:54.235790 11710 raft_consensus.cc:2726] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20240502 20:46:54.236617 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 8 candidate_status { last_received { term: 5 index: 13 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.236716 9286 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.237406 9286 raft_consensus.cc:2396] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 8 because replica has last-logged OpId of term: 6 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 13.
I20240502 20:46:54.237773 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20698b2531674eefb9df3125a86653ba" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 9 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.237872 9286 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.237824 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 8 candidate_status { last_received { term: 5 index: 11 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.238065 9287 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.238143 9355 leader_election.cc:302] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f
I20240502 20:46:54.238343 11710 raft_consensus.cc:2726] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20240502 20:46:54.238613 9355 leader_election.cc:302] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e
I20240502 20:46:54.238655 9286 raft_consensus.cc:2454] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 9.
I20240502 20:46:54.238816 11710 raft_consensus.cc:2781] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Leader election won for term 8
I20240502 20:46:54.238902 11710 raft_consensus.cc:686] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.239032 9287 raft_consensus.cc:2396] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 8 because replica has last-logged OpId of term: 6 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 11.
I20240502 20:46:54.239032 11710 consensus_queue.cc:237] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 5.10, Last appended by leader: 10, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.240223 11710 consensus_queue.cc:1035] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20240502 20:46:54.240329 11710 consensus_queue.cc:1225] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Peer 4b3cb6ad00954dc5a1270dc392870b4f log is divergent from this leader: its last log entry 6.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20240502 20:46:54.240305 9355 leader_election.cc:302] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e
I20240502 20:46:54.240523 11889 raft_consensus.cc:2781] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Leader election won for term 9
I20240502 20:46:54.240495 9142 catalog_manager.cc:5484] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 7 to 8, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 8 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:54.240595 11889 raft_consensus.cc:686] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 9 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.240685 11889 consensus_queue.cc:237] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 6.16, Last appended by leader: 16, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.241657 9287 pending_rounds.cc:76] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f: Aborting all ops after (but not including) 16
I20240502 20:46:54.242007 9287 pending_rounds.cc:97] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f: Aborting uncommitted NO_OP operation due to leader change: 6.17
I20240502 20:46:54.242053 9287 raft_consensus.cc:2866] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 20:46:54.243481 9142 catalog_manager.cc:5484] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 8 to 9, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 9 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
W20240502 20:46:54.253916 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40118: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
W20240502 20:46:54.254810 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59834: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:54.257254 9404 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
W20240502 20:46:54.264873 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:54.264925 9286 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:54.265018 9286 raft_consensus.cc:482] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.265067 9286 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.266072 9286 raft_consensus.cc:504] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.266726 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.266837 9686 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.267717 9686 raft_consensus.cc:2454] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 8.
I20240502 20:46:54.268046 9223 leader_election.cc:302] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:54.268394 11819 raft_consensus.cc:2781] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Leader election won for term 8
I20240502 20:46:54.268469 11819 raft_consensus.cc:686] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.268584 11819 consensus_queue.cc:237] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.271895 9286 leader_election.cc:288] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 8 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:54.271971 9687 raft_consensus.cc:1261] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 7 index: 18. Preceding OpId from leader: term: 9 index: 20. (index mismatch)
I20240502 20:46:54.272435 11889 consensus_queue.cc:1035] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20240502 20:46:54.272907 9142 catalog_manager.cc:5484] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 7 to 8, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 8 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:54.273128 9555 raft_consensus.cc:1261] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 8 index: 19. Preceding OpId from leader: term: 9 index: 20. (index mismatch)
W20240502 20:46:54.275306 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51610: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:54.276540 11710 consensus_queue.cc:1035] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20240502 20:46:54.276641 11710 consensus_queue.cc:1225] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Peer 190c1d3968d949bbbaa27a51a2b91d8e log is divergent from this leader: its last log entry 8.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
W20240502 20:46:54.277814 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40188: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:54.278038 9555 pending_rounds.cc:76] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e: Aborting all ops after (but not including) 18
I20240502 20:46:54.278086 9555 pending_rounds.cc:97] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e: Aborting uncommitted NO_OP operation due to leader change: 8.19
I20240502 20:46:54.278116 9555 raft_consensus.cc:2866] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 20:46:54.280129 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.280238 9424 raft_consensus.cc:3032] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 LEADER]: Stepping down as leader of term 7
I20240502 20:46:54.280277 9424 raft_consensus.cc:729] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
W20240502 20:46:54.280280 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:54.280400 9424 consensus_queue.cc:260] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.280529 9424 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.281502 9424 raft_consensus.cc:2396] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20240502 20:46:54.282944 9287 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:54.283105 9287 raft_consensus.cc:482] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.283354 9287 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.283947 11300 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 20:46:54.284307 9287 raft_consensus.cc:504] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.284631 9287 leader_election.cc:288] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 9 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:54.285122 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.285224 9687 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.286190 9687 raft_consensus.cc:2396] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20240502 20:46:54.287173 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.287282 9555 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.288303 9555 raft_consensus.cc:2454] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 9.
I20240502 20:46:54.288543 9223 leader_election.cc:302] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters: 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:54.288687 10895 raft_consensus.cc:2781] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Leader election won for term 9
I20240502 20:46:54.288767 10895 raft_consensus.cc:686] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.288875 10895 consensus_queue.cc:237] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.290307 9139 catalog_manager.cc:5484] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 7 to 9, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 9 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:54.291517 9286 raft_consensus.cc:1261] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 6 index: 20. Preceding OpId from leader: term: 7 index: 22. (index mismatch)
I20240502 20:46:54.291659 9287 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:54.291788 9687 raft_consensus.cc:1261] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Refusing update from remote peer 88a19090795d47c2a1ea1dd979fb62bc: Log matching property violated. Preceding OpId in replica: term: 6 index: 20. Preceding OpId from leader: term: 7 index: 22. (index mismatch)
I20240502 20:46:54.292150 11712 consensus_queue.cc:1035] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20240502 20:46:54.292291 9287 raft_consensus.cc:482] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.292376 9287 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.292423 11712 consensus_queue.cc:1035] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20240502 20:46:54.293155 9287 raft_consensus.cc:504] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.293717 9287 leader_election.cc:288] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 10 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:54.294188 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.294301 9424 raft_consensus.cc:3032] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 9 LEADER]: Stepping down as leader of term 9
I20240502 20:46:54.294349 9424 raft_consensus.cc:729] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.294500 9424 consensus_queue.cc:260] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.294639 9424 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.294797 9287 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:54.294942 9287 raft_consensus.cc:482] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.295046 9287 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.296231 9287 raft_consensus.cc:504] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
W20240502 20:46:54.296447 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59728: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:54.296739 9424 raft_consensus.cc:2396] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20240502 20:46:54.296837 9287 leader_election.cc:288] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 9 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.296895 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 33 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.297012 9553 raft_consensus.cc:3032] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Stepping down as leader of term 8
I20240502 20:46:54.297058 9553 raft_consensus.cc:729] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.297191 9553 consensus_queue.cc:260] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 8.33, Last appended by leader: 33, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.297362 9553 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.297667 9287 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:54.297811 9287 raft_consensus.cc:482] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.297891 9287 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.298100 9553 raft_consensus.cc:2454] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 9.
I20240502 20:46:54.298522 9223 leader_election.cc:302] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:54.298784 9287 raft_consensus.cc:504] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.298864 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 33 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.298964 9687 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.299211 9287 leader_election.cc:288] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 8 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.299283 11819 raft_consensus.cc:2781] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Leader election won for term 9
I20240502 20:46:54.299360 11819 raft_consensus.cc:686] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.299468 11819 consensus_queue.cc:237] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 8.33, Last appended by leader: 33, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.299818 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 41 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.299906 9688 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.300019 9687 raft_consensus.cc:2454] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 9.
I20240502 20:46:54.300208 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.300304 9555 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.300745 9688 raft_consensus.cc:2454] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 8.
I20240502 20:46:54.301137 9555 raft_consensus.cc:2454] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 10.
I20240502 20:46:54.301362 9287 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:54.301513 9287 raft_consensus.cc:482] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.301617 9287 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.301400 9223 leader_election.cc:302] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:54.302095 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 8 candidate_status { last_received { term: 6 index: 41 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.302110 9223 leader_election.cc:302] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters: 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:54.302198 9555 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.302268 10895 raft_consensus.cc:2781] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Leader election won for term 8
I20240502 20:46:54.302353 10895 raft_consensus.cc:686] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.302399 11739 raft_consensus.cc:2781] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Leader election won for term 10
I20240502 20:46:54.302568 11739 raft_consensus.cc:686] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.302592 10895 consensus_queue.cc:237] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 41, Committed index: 41, Last appended: 6.41, Last appended by leader: 41, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.302695 11739 consensus_queue.cc:237] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
W20240502 20:46:54.302896 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51626: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:54.303503 9287 raft_consensus.cc:504] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.303851 9287 leader_election.cc:288] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 10 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:54.304162 9555 raft_consensus.cc:2454] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 8.
I20240502 20:46:54.304895 9287 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:54.305080 9287 raft_consensus.cc:482] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.305148 9287 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.305580 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 10 candidate_status { last_received { term: 7 index: 12 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.305843 9555 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.305965 9287 raft_consensus.cc:504] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.306346 9287 leader_election.cc:288] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 9 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.306686 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.306792 9688 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.307387 9287 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:54.307523 9287 raft_consensus.cc:482] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.307575 9287 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.307607 9688 raft_consensus.cc:2454] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 9.
I20240502 20:46:54.307886 9223 leader_election.cc:302] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:54.307703 9555 raft_consensus.cc:2396] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 10 because replica has last-logged OpId of term: 8 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 12.
I20240502 20:46:54.308347 9287 raft_consensus.cc:504] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.308660 9287 leader_election.cc:288] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 10 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:54.308935 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.309036 9555 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.309608 9139 catalog_manager.cc:5484] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 8 to 9, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 9 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:54.309762 9555 raft_consensus.cc:2454] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 10.
I20240502 20:46:54.309813 11903 raft_consensus.cc:2781] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Leader election won for term 9
I20240502 20:46:54.309906 11903 raft_consensus.cc:686] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.310024 9223 leader_election.cc:302] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:54.310020 11903 consensus_queue.cc:237] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.310079 9139 catalog_manager.cc:5484] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 9 to 10, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 10 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:54.310416 9139 catalog_manager.cc:5484] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 6 to 8, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 8 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:54.310596 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.310701 9420 raft_consensus.cc:3032] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 8 LEADER]: Stepping down as leader of term 8
I20240502 20:46:54.310748 9420 raft_consensus.cc:729] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.310848 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 10 candidate_status { last_received { term: 7 index: 12 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.310873 9420 consensus_queue.cc:260] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.310950 9424 raft_consensus.cc:3032] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 9 LEADER]: Stepping down as leader of term 9
I20240502 20:46:54.310993 9424 raft_consensus.cc:729] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.311012 9420 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.311118 9424 consensus_queue.cc:260] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 9.13, Last appended by leader: 13, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.311254 9424 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.311744 9420 raft_consensus.cc:2396] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20240502 20:46:54.312122 9424 raft_consensus.cc:2396] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 10 because replica has last-logged OpId of term: 9 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 12.
I20240502 20:46:54.312517 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.310225 11739 raft_consensus.cc:2781] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Leader election won for term 10
I20240502 20:46:54.312609 9420 raft_consensus.cc:3032] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 9 LEADER]: Stepping down as leader of term 9
I20240502 20:46:54.312644 9420 raft_consensus.cc:729] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.312633 11739 raft_consensus.cc:686] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.312752 11739 consensus_queue.cc:237] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.312758 9420 consensus_queue.cc:260] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.312889 9420 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.313740 9420 raft_consensus.cc:2396] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20240502 20:46:54.312162 9286 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3315525f1849462fbf49a285f424ea51"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:54.314044 9286 raft_consensus.cc:482] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.314102 9286 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.314735 9286 raft_consensus.cc:504] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.315029 9286 leader_election.cc:288] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 10 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:54.315317 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3315525f1849462fbf49a285f424ea51" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 10 candidate_status { last_received { term: 7 index: 40 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.315413 9555 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.315542 9286 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fe22224142984d69a2def286584b9aaa"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:54.315642 9286 raft_consensus.cc:482] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.315696 9286 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.316151 9555 raft_consensus.cc:2396] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 10 because replica has last-logged OpId of term: 8 index: 41, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 40.
I20240502 20:46:54.316470 9286 raft_consensus.cc:504] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.316736 9286 leader_election.cc:288] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 9 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:54.316998 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe22224142984d69a2def286584b9aaa" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 9 candidate_status { last_received { term: 6 index: 20 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.317087 9688 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.318099 9222 leader_election.cc:302] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:54.318495 11903 raft_consensus.cc:2726] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20240502 20:46:54.318562 9286 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:54.318652 9286 raft_consensus.cc:482] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.318708 9286 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.320266 9142 catalog_manager.cc:5484] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 8 to 9, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 9 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:54.320470 9142 catalog_manager.cc:5484] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 9 to 10, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 10 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:54.320870 9688 raft_consensus.cc:2454] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 9.
I20240502 20:46:54.321171 9553 raft_consensus.cc:1261] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 6 index: 40. Preceding OpId from leader: term: 8 index: 43. (index mismatch)
I20240502 20:46:54.321642 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3315525f1849462fbf49a285f424ea51" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 10 candidate_status { last_received { term: 7 index: 40 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.321769 9420 raft_consensus.cc:3032] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 LEADER]: Stepping down as leader of term 9
I20240502 20:46:54.321810 9420 raft_consensus.cc:729] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.321624 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe22224142984d69a2def286584b9aaa" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 9 candidate_status { last_received { term: 6 index: 20 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.321908 9424 raft_consensus.cc:3032] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 8 LEADER]: Stepping down as leader of term 8
I20240502 20:46:54.321918 9420 consensus_queue.cc:260] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 9.41, Last appended by leader: 41, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.322081 9420 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.322926 9420 raft_consensus.cc:2396] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 10 because replica has last-logged OpId of term: 9 index: 41, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 40.
I20240502 20:46:54.323120 9223 leader_election.cc:302] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:54.323266 10895 raft_consensus.cc:2781] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Leader election won for term 9
I20240502 20:46:54.323338 10895 raft_consensus.cc:686] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.323429 10895 consensus_queue.cc:237] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 6.20, Last appended by leader: 20, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.324755 10895 consensus_queue.cc:1035] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 42, Last known committed idx: 40, Time since last communication: 0.000s
I20240502 20:46:54.325922 9139 catalog_manager.cc:5484] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 8 to 9, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 9 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:54.326262 9688 raft_consensus.cc:1261] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 6 index: 41. Preceding OpId from leader: term: 8 index: 43. (index mismatch)
W20240502 20:46:54.328171 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59834: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:54.328841 9222 leader_election.cc:302] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:54.329448 11903 raft_consensus.cc:2726] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20240502 20:46:54.329658 9289 raft_consensus.cc:1261] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 7 index: 39. Preceding OpId from leader: term: 8 index: 41. (index mismatch)
I20240502 20:46:54.330101 9688 raft_consensus.cc:1261] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 7 index: 39. Preceding OpId from leader: term: 8 index: 41. (index mismatch)
I20240502 20:46:54.321959 9424 raft_consensus.cc:729] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.330540 11532 consensus_queue.cc:1035] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 40, Last known committed idx: 39, Time since last communication: 0.000s
I20240502 20:46:54.330674 9286 raft_consensus.cc:504] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.330993 9424 consensus_queue.cc:260] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 8.21, Last appended by leader: 21, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.331137 9424 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.331496 10895 consensus_queue.cc:1035] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 42, Last known committed idx: 41, Time since last communication: 0.000s
I20240502 20:46:54.331624 9287 raft_consensus.cc:1226] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Rejecting Update request from peer 88a19090795d47c2a1ea1dd979fb62bc for earlier term 8. Current term is 9. Ops: []
I20240502 20:46:54.332351 11532 consensus_queue.cc:1035] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 40, Last known committed idx: 39, Time since last communication: 0.000s
I20240502 20:46:54.332512 11889 consensus_queue.cc:1046] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: INVALID_TERM, Last received: 8.21, Next index: 22, Last known committed idx: 20, Time since last communication: 0.000s
I20240502 20:46:54.332665 11889 raft_consensus.cc:3032] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 LEADER]: Stepping down as leader of term 8
I20240502 20:46:54.332700 11889 raft_consensus.cc:729] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.333213 11889 consensus_queue.cc:260] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 8.22, Last appended by leader: 22, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.333380 11889 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.334822 9286 leader_election.cc:288] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 9 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:54.335465 9424 raft_consensus.cc:2396] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 9 because replica has last-logged OpId of term: 8 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 20.
I20240502 20:46:54.340616 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 21 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.340716 9688 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.341559 9688 raft_consensus.cc:2396] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 9 because replica has last-logged OpId of term: 8 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 21.
I20240502 20:46:54.335848 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 21 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.347699 9420 raft_consensus.cc:2396] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 9 because replica has last-logged OpId of term: 8 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 21.
I20240502 20:46:54.347997 9222 leader_election.cc:302] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f; no voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:54.348016 9287 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
W20240502 20:46:54.346132 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55522: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:54.348181 10895 raft_consensus.cc:2726] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20240502 20:46:54.348336 9287 raft_consensus.cc:482] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.348523 9287 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.349377 9287 raft_consensus.cc:504] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
W20240502 20:46:54.350291 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51610: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:54.350378 9287 leader_election.cc:288] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 8 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.350607 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 8 candidate_status { last_received { term: 5 index: 13 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.350718 9553 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.351651 9553 raft_consensus.cc:2454] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 8.
I20240502 20:46:54.351754 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 8 candidate_status { last_received { term: 5 index: 13 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.351858 9687 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.351961 9223 leader_election.cc:302] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:54.352156 10895 raft_consensus.cc:2781] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Leader election won for term 8
I20240502 20:46:54.352249 10895 raft_consensus.cc:686] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.352362 10895 consensus_queue.cc:237] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 5.13, Last appended by leader: 13, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.352792 9687 raft_consensus.cc:2396] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 8 because replica has last-logged OpId of term: 6 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 13.
I20240502 20:46:54.353415 9287 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:54.353540 9287 raft_consensus.cc:482] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.353605 9287 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Advancing to term 9
W20240502 20:46:54.354807 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40188: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:54.354961 9287 raft_consensus.cc:504] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.355219 9139 catalog_manager.cc:5484] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 6 to 8, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 8 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:54.355455 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 33 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.355557 9553 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.357070 9553 raft_consensus.cc:2454] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 9.
W20240502 20:46:54.357491 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59728: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:54.357599 9223 leader_election.cc:302] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:54.357723 11739 raft_consensus.cc:2781] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Leader election won for term 9
I20240502 20:46:54.357803 11739 raft_consensus.cc:686] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.357913 11739 consensus_queue.cc:237] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 7.33, Last appended by leader: 33, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.359848 9139 catalog_manager.cc:5484] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 7 to 9, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 9 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
W20240502 20:46:54.360271 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
W20240502 20:46:54.362488 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55522: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:54.362617 9287 leader_election.cc:288] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 9 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:54.363169 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 9 candidate_status { last_received { term: 7 index: 33 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.363286 9424 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.364092 9424 raft_consensus.cc:2454] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 9.
I20240502 20:46:54.364437 9287 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:54.364528 9287 raft_consensus.cc:482] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.364568 9287 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Advancing to term 8
W20240502 20:46:54.364675 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:54.365259 9287 raft_consensus.cc:504] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.365550 9287 leader_election.cc:288] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 8 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.365808 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 8 candidate_status { last_received { term: 5 index: 17 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.365911 9553 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.366218 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 8 candidate_status { last_received { term: 5 index: 17 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.366226 9287 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:54.366322 9687 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.366356 9287 raft_consensus.cc:482] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.366400 9287 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.366771 9553 raft_consensus.cc:2454] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 8.
I20240502 20:46:54.367033 9223 leader_election.cc:302] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:54.367125 9287 raft_consensus.cc:504] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.367177 10895 raft_consensus.cc:2781] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Leader election won for term 8
I20240502 20:46:54.367251 10895 raft_consensus.cc:686] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.367353 10895 consensus_queue.cc:237] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 5.17, Last appended by leader: 17, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.367430 9287 leader_election.cc:288] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 9 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:54.367566 9687 raft_consensus.cc:2396] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 8 because replica has last-logged OpId of term: 6 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 17.
I20240502 20:46:54.368261 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 9 candidate_status { last_received { term: 5 index: 10 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.368366 9553 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.368458 9287 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d92bfee8fabd46a28ffcb7e319879406"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:54.368551 9287 raft_consensus.cc:482] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.368598 9287 raft_consensus.cc:3037] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.368799 9139 catalog_manager.cc:5484] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 6 to 8, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 8 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:54.369060 9553 raft_consensus.cc:2396] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 9 because replica has last-logged OpId of term: 7 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 10.
I20240502 20:46:54.369351 9287 raft_consensus.cc:504] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.369390 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 9 candidate_status { last_received { term: 5 index: 10 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.369478 9424 raft_consensus.cc:3032] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 LEADER]: Stepping down as leader of term 8
I20240502 20:46:54.369511 9424 raft_consensus.cc:729] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.369657 9287 leader_election.cc:288] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 8 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.369640 9424 consensus_queue.cc:260] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.11, Last appended by leader: 11, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.369782 9424 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.370692 9424 raft_consensus.cc:2396] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 9 because replica has last-logged OpId of term: 8 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 10.
W20240502 20:46:54.370815 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59752: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:54.371233 9222 leader_election.cc:302] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:54.371402 9287 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:54.371428 10895 raft_consensus.cc:2726] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20240502 20:46:54.371604 9287 raft_consensus.cc:482] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
W20240502 20:46:54.371634 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51610: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:54.371656 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d92bfee8fabd46a28ffcb7e319879406" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 26 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.371688 9287 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.371734 9424 raft_consensus.cc:3032] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 LEADER]: Stepping down as leader of term 7
I20240502 20:46:54.371779 9424 raft_consensus.cc:729] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.371909 9424 consensus_queue.cc:260] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 7.26, Last appended by leader: 26, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.372071 9424 raft_consensus.cc:3037] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.372393 9287 raft_consensus.cc:504] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.372377 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d92bfee8fabd46a28ffcb7e319879406" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 8 candidate_status { last_received { term: 7 index: 26 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.372504 9687 raft_consensus.cc:3037] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 7 FOLLOWER]: Advancing to term 8
I20240502 20:46:54.372839 9424 raft_consensus.cc:2454] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 8.
I20240502 20:46:54.373104 9287 leader_election.cc:288] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 9 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.373273 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 9 candidate_status { last_received { term: 6 index: 13 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.373330 9687 raft_consensus.cc:2454] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 8.
I20240502 20:46:54.373366 9686 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.373715 9286 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:54.373819 9286 raft_consensus.cc:482] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.373873 9286 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.374215 9686 raft_consensus.cc:2454] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 9.
I20240502 20:46:54.373132 9222 leader_election.cc:302] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:54.374503 10895 raft_consensus.cc:2781] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Leader election won for term 8
I20240502 20:46:54.374583 10895 raft_consensus.cc:686] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.374680 9286 raft_consensus.cc:504] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.374745 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 9 candidate_status { last_received { term: 6 index: 13 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.374830 9424 raft_consensus.cc:3032] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 8 LEADER]: Stepping down as leader of term 8
I20240502 20:46:54.374868 9424 raft_consensus.cc:729] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.374928 9223 leader_election.cc:302] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:54.375016 9286 leader_election.cc:288] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 9 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.374980 9424 consensus_queue.cc:260] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 8.14, Last appended by leader: 14, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.375100 9424 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.374680 10895 consensus_queue.cc:237] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 7.26, Last appended by leader: 26, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.375258 11739 raft_consensus.cc:2781] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Leader election won for term 9
I20240502 20:46:54.375336 11739 raft_consensus.cc:686] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.375427 11739 consensus_queue.cc:237] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 6.13, Last appended by leader: 13, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.376044 9424 raft_consensus.cc:2396] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 9 because replica has last-logged OpId of term: 8 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 13.
I20240502 20:46:54.376421 9286 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "20698b2531674eefb9df3125a86653ba"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:54.376441 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 42 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.376518 9286 raft_consensus.cc:482] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.376525 9553 raft_consensus.cc:3032] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Stepping down as leader of term 8
I20240502 20:46:54.376559 9286 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.376555 9553 raft_consensus.cc:729] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.376668 9553 consensus_queue.cc:260] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 42, Committed index: 42, Last appended: 8.42, Last appended by leader: 42, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.376803 9553 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.377071 9286 raft_consensus.cc:504] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.377359 9286 leader_election.cc:288] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 10 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:54.377487 9553 raft_consensus.cc:2454] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 9.
I20240502 20:46:54.377599 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20698b2531674eefb9df3125a86653ba" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 10 candidate_status { last_received { term: 3 index: 14 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.377688 9424 raft_consensus.cc:3032] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 9 LEADER]: Stepping down as leader of term 9
I20240502 20:46:54.377722 9424 raft_consensus.cc:729] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.377841 9424 consensus_queue.cc:260] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 9.17, Last appended by leader: 17, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.377966 9424 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.378072 9223 leader_election.cc:302] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:54.378136 9139 catalog_manager.cc:5484] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 7 to 8, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 8 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:54.378249 9286 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:54.378355 9286 raft_consensus.cc:482] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.378348 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20698b2531674eefb9df3125a86653ba" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 10 candidate_status { last_received { term: 3 index: 14 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.378401 9286 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.378433 9553 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.378739 9424 raft_consensus.cc:2396] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 10 because replica has last-logged OpId of term: 9 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 14.
I20240502 20:46:54.378260 11903 raft_consensus.cc:2781] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Leader election won for term 9
I20240502 20:46:54.379235 11903 raft_consensus.cc:686] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.379366 11903 consensus_queue.cc:237] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 42, Committed index: 42, Last appended: 8.42, Last appended by leader: 42, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.379135 9286 raft_consensus.cc:504] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.380264 9286 leader_election.cc:288] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 10 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:54.380882 9286 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:54.380983 9286 raft_consensus.cc:482] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.381042 9286 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Advancing to term 9
W20240502 20:46:54.381145 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
W20240502 20:46:54.381182 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51626: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:54.381561 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 10 candidate_status { last_received { term: 7 index: 14 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.381659 9424 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.381778 9286 raft_consensus.cc:504] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.382088 9286 leader_election.cc:288] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 9 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.382087 9142 catalog_manager.cc:5484] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 8 to 9, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 9 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:54.382277 9142 catalog_manager.cc:5484] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 8 to 9, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 9 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:54.382398 9424 raft_consensus.cc:2454] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 10.
W20240502 20:46:54.382442 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59728: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:54.382742 9550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 9 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.382844 9550 raft_consensus.cc:3032] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Stepping down as leader of term 8
I20240502 20:46:54.382882 9550 raft_consensus.cc:729] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.383019 9553 raft_consensus.cc:2396] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 10 because replica has last-logged OpId of term: 8 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 14.
I20240502 20:46:54.383006 9550 consensus_queue.cc:260] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 8.17, Last appended by leader: 17, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.383126 9550 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.383519 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 9 candidate_status { last_received { term: 8 index: 42 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.383625 9686 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.383534 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 9 candidate_status { last_received { term: 6 index: 16 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.383772 9687 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.384052 9550 raft_consensus.cc:2396] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 9 because replica has last-logged OpId of term: 8 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 16.
I20240502 20:46:54.384395 9686 raft_consensus.cc:2454] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 9.
I20240502 20:46:54.383210 9286 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45"
dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
from {username='slave'} at 127.0.0.1:59772
I20240502 20:46:54.384641 9286 raft_consensus.cc:482] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.384668 9687 raft_consensus.cc:2454] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 9.
I20240502 20:46:54.384689 9286 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.382643 9222 leader_election.cc:302] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:54.384940 11739 raft_consensus.cc:2781] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Leader election won for term 10
I20240502 20:46:54.385025 11739 raft_consensus.cc:686] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.385138 11739 consensus_queue.cc:237] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 7.14, Last appended by leader: 14, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.385061 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 10 candidate_status { last_received { term: 7 index: 14 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.385253 9555 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.383360 9223 leader_election.cc:302] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:54.385893 11903 raft_consensus.cc:2726] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20240502 20:46:54.386190 9223 leader_election.cc:302] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters: 190c1d3968d949bbbaa27a51a2b91d8e
I20240502 20:46:54.386340 9555 raft_consensus.cc:2396] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 10 because replica has last-logged OpId of term: 8 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 14.
I20240502 20:46:54.386526 11903 raft_consensus.cc:2781] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Leader election won for term 9
I20240502 20:46:54.386611 11903 raft_consensus.cc:686] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.386638 9286 raft_consensus.cc:504] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.386623 9142 catalog_manager.cc:5484] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 8 to 10, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 10 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:54.386713 11903 consensus_queue.cc:237] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 6.16, Last appended by leader: 16, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.387024 9286 leader_election.cc:288] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 9 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.387490 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 9 candidate_status { last_received { term: 6 index: 11 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.387486 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45" candidate_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" candidate_term: 9 candidate_status { last_received { term: 6 index: 11 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.387580 9424 raft_consensus.cc:3032] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 LEADER]: Stepping down as leader of term 8
I20240502 20:46:54.387667 9687 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.387626 9424 raft_consensus.cc:729] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.388150 9424 consensus_queue.cc:260] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.12, Last appended by leader: 12, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.388280 9424 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.389034 9424 raft_consensus.cc:2396] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b3cb6ad00954dc5a1270dc392870b4f for term 9 because replica has last-logged OpId of term: 8 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 11.
I20240502 20:46:54.389050 9687 raft_consensus.cc:2454] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3cb6ad00954dc5a1270dc392870b4f in term 9.
I20240502 20:46:54.389360 9223 leader_election.cc:302] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:54.389292 9142 catalog_manager.cc:5484] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 8 to 9, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 9 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:54.389520 11739 raft_consensus.cc:2781] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Leader election won for term 9
I20240502 20:46:54.389586 11739 raft_consensus.cc:686] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 LEADER]: Becoming Leader. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.389693 11739 consensus_queue.cc:237] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.11, Last appended by leader: 11, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
W20240502 20:46:54.390633 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:54.392244 9142 catalog_manager.cc:5484] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f reported cstate change: term changed from 8 to 9, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65). New cstate: current_term: 9 leader_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
W20240502 20:46:54.393488 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55522: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
W20240502 20:46:54.400975 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59752: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:54.403108 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:54.403213 9687 raft_consensus.cc:482] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.403262 9687 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.404007 9687 raft_consensus.cc:504] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.404563 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.404682 9555 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.404731 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.404824 9286 raft_consensus.cc:3032] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 LEADER]: Stepping down as leader of term 9
I20240502 20:46:54.404860 9286 raft_consensus.cc:729] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.404980 9286 consensus_queue.cc:260] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.405090 9286 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.405567 9555 raft_consensus.cc:2454] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 10.
I20240502 20:46:54.405910 9286 raft_consensus.cc:2396] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20240502 20:46:54.406260 9687 leader_election.cc:288] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 10 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
W20240502 20:46:54.406598 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51610: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:54.406653 9623 leader_election.cc:302] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters: 4b3cb6ad00954dc5a1270dc392870b4f
I20240502 20:46:54.406903 11624 raft_consensus.cc:2781] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Leader election won for term 10
I20240502 20:46:54.406993 11624 raft_consensus.cc:686] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 10 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.407101 11624 consensus_queue.cc:237] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.407888 9555 raft_consensus.cc:1261] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 8 index: 15. (index mismatch)
I20240502 20:46:54.407888 9687 raft_consensus.cc:1261] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Refusing update from remote peer 4b3cb6ad00954dc5a1270dc392870b4f: Log matching property violated. Preceding OpId in replica: term: 6 index: 13. Preceding OpId from leader: term: 8 index: 15. (index mismatch)
I20240502 20:46:54.408196 11739 consensus_queue.cc:1035] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 20:46:54.408294 11739 consensus_queue.cc:1225] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Peer 64136bd82a30486e9d56fc28da4bd448 log is divergent from this leader: its last log entry 6.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20240502 20:46:54.408529 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:54.408725 9142 catalog_manager.cc:5484] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 9 to 10, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 10 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:54.408879 9686 pending_rounds.cc:76] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448: Aborting all ops after (but not including) 11
I20240502 20:46:54.408922 9686 pending_rounds.cc:97] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448: Aborting uncommitted NO_OP operation due to leader change: 3.12
I20240502 20:46:54.408947 9686 raft_consensus.cc:2866] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 20:46:54.408986 9686 pending_rounds.cc:97] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448: Aborting uncommitted NO_OP operation due to leader change: 6.13
I20240502 20:46:54.409008 9686 raft_consensus.cc:2866] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 20:46:54.409521 9687 raft_consensus.cc:482] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.409587 9687 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.410164 11739 consensus_queue.cc:1035] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 20:46:54.410363 11697 mvcc.cc:205] Tried to move back new op lower bound from 7023340807811035136 to 7023340807588540416. Current Snapshot: MvccSnapshot[applied={T|T < 7023340807811035136}]
W20240502 20:46:54.412375 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:54.412474 9687 raft_consensus.cc:504] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.412993 9687 leader_election.cc:288] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 9 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.413045 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 9 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.413152 9286 raft_consensus.cc:3032] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 LEADER]: Stepping down as leader of term 8
I20240502 20:46:54.413188 9286 raft_consensus.cc:729] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.413302 9286 consensus_queue.cc:260] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.413440 9286 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.414235 9286 raft_consensus.cc:2396] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20240502 20:46:54.414532 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 9 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.414628 9424 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.415443 9424 raft_consensus.cc:2396] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20240502 20:46:54.416028 9622 leader_election.cc:302] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:54.416241 11839 raft_consensus.cc:2726] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20240502 20:46:54.416579 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:54.416905 9687 raft_consensus.cc:482] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.416986 9687 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Advancing to term 10
W20240502 20:46:54.418208 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59728: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:54.417708 9687 raft_consensus.cc:504] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.418785 9550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 10 candidate_status { last_received { term: 9 index: 40 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.418877 9550 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.419488 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 10 candidate_status { last_received { term: 9 index: 40 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.419579 9424 raft_consensus.cc:3032] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 LEADER]: Stepping down as leader of term 9
I20240502 20:46:54.419616 9424 raft_consensus.cc:729] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.419647 9550 raft_consensus.cc:2454] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 10.
I20240502 20:46:54.419742 9424 consensus_queue.cc:260] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 9.40, Last appended by leader: 40, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.419914 9424 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.419917 9623 leader_election.cc:302] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:54.420064 11839 raft_consensus.cc:2781] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Leader election won for term 10
I20240502 20:46:54.420132 11839 raft_consensus.cc:686] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 10 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.420244 11839 consensus_queue.cc:237] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 9.40, Last appended by leader: 40, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.420751 9424 raft_consensus.cc:2454] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 10.
I20240502 20:46:54.421367 9687 leader_election.cc:288] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 10 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:54.421859 9142 catalog_manager.cc:5484] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 9 to 10, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 10 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:54.422520 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:54.422653 9687 raft_consensus.cc:482] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.422741 9687 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.424757 9687 raft_consensus.cc:504] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
W20240502 20:46:54.425120 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59834: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
W20240502 20:46:54.425479 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40096: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:54.425520 9550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 10 candidate_status { last_received { term: 8 index: 33 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.425613 9550 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.426437 9550 raft_consensus.cc:2454] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 10.
I20240502 20:46:54.426924 9623 leader_election.cc:302] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:54.427078 11758 raft_consensus.cc:2781] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Leader election won for term 10
I20240502 20:46:54.427161 11758 raft_consensus.cc:686] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 10 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.427261 11758 consensus_queue.cc:237] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 8.33, Last appended by leader: 33, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.427866 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 10 candidate_status { last_received { term: 8 index: 33 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.427963 9286 raft_consensus.cc:3032] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 LEADER]: Stepping down as leader of term 9
I20240502 20:46:54.428001 9286 raft_consensus.cc:729] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.428112 9286 consensus_queue.cc:260] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 9.34, Last appended by leader: 34, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.428222 9286 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.428576 9142 catalog_manager.cc:5484] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 9 to 10, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 10 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:54.428984 9286 raft_consensus.cc:2396] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 10 because replica has last-logged OpId of term: 9 index: 34, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 33.
I20240502 20:46:54.429344 9687 leader_election.cc:288] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 10 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.430048 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:54.430141 9687 raft_consensus.cc:482] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.430218 9687 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.431023 9687 raft_consensus.cc:504] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.431569 9550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 9 candidate_status { last_received { term: 8 index: 43 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
W20240502 20:46:54.431603 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55522: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:54.431592 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 9 candidate_status { last_received { term: 8 index: 43 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.431659 9550 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.431680 9286 raft_consensus.cc:3032] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 LEADER]: Stepping down as leader of term 8
I20240502 20:46:54.431713 9286 raft_consensus.cc:729] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.431840 9286 consensus_queue.cc:260] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 43, Committed index: 43, Last appended: 8.43, Last appended by leader: 43, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.431990 9286 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.432382 9550 raft_consensus.cc:2454] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 9.
I20240502 20:46:54.432732 9623 leader_election.cc:302] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:54.432775 9286 raft_consensus.cc:2454] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 9.
I20240502 20:46:54.432878 11839 raft_consensus.cc:2781] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Leader election won for term 9
I20240502 20:46:54.432943 11839 raft_consensus.cc:686] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 9 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.433038 11839 consensus_queue.cc:237] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 43, Committed index: 43, Last appended: 8.43, Last appended by leader: 43, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
W20240502 20:46:54.433670 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55548: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:54.434522 9687 leader_election.cc:288] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 9 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.434732 9142 catalog_manager.cc:5484] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 8 to 9, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 9 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:54.435348 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:54.435446 9687 raft_consensus.cc:482] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.435494 9687 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Advancing to term 10
W20240502 20:46:54.435681 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59752: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:54.436465 9687 raft_consensus.cc:504] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.437121 9550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 10 candidate_status { last_received { term: 8 index: 42 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.437214 9550 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.437955 9550 raft_consensus.cc:2454] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 10.
I20240502 20:46:54.438014 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 10 candidate_status { last_received { term: 8 index: 42 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.438099 9286 raft_consensus.cc:3032] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 LEADER]: Stepping down as leader of term 9
I20240502 20:46:54.438129 9286 raft_consensus.cc:729] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.438232 9286 consensus_queue.cc:260] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 42, Committed index: 42, Last appended: 9.43, Last appended by leader: 43, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.438345 9286 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.438333 9623 leader_election.cc:302] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:54.438503 11839 raft_consensus.cc:2781] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Leader election won for term 10
I20240502 20:46:54.438575 11839 raft_consensus.cc:686] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 10 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.438681 11839 consensus_queue.cc:237] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 42, Committed index: 42, Last appended: 8.42, Last appended by leader: 42, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
W20240502 20:46:54.438712 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51626: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:54.439158 9286 raft_consensus.cc:2396] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 10 because replica has last-logged OpId of term: 9 index: 43, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 42.
I20240502 20:46:54.439152 9687 leader_election.cc:288] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 10 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:54.440104 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:54.440200 9687 raft_consensus.cc:482] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.440266 9687 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.440843 9687 raft_consensus.cc:504] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.441097 9687 leader_election.cc:288] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 9 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:54.441378 9550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.441476 9550 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.442066 9550 raft_consensus.cc:2396] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20240502 20:46:54.442536 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:54.442628 9687 raft_consensus.cc:482] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.442683 9687 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.442870 9142 catalog_manager.cc:5484] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 9 to 10, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 10 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:54.443248 9687 raft_consensus.cc:504] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.443331 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.443439 9424 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.443568 9687 leader_election.cc:288] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 10 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.443964 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.444056 9420 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.444119 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea141ae57324a879dfd26cbeea9aead" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.444223 9286 raft_consensus.cc:3032] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 LEADER]: Stepping down as leader of term 9
I20240502 20:46:54.444275 9286 raft_consensus.cc:729] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.444327 9424 raft_consensus.cc:2454] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 9.
I20240502 20:46:54.444406 9286 consensus_queue.cc:260] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.444522 9286 raft_consensus.cc:3037] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.444598 9622 leader_election.cc:302] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e
I20240502 20:46:54.444841 11758 raft_consensus.cc:2781] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Leader election won for term 9
I20240502 20:46:54.444916 11758 raft_consensus.cc:686] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 9 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.444979 9420 raft_consensus.cc:2396] T dea141ae57324a879dfd26cbeea9aead P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20240502 20:46:54.445019 11758 consensus_queue.cc:237] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.445237 9286 raft_consensus.cc:2396] T dea141ae57324a879dfd26cbeea9aead P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20240502 20:46:54.446054 9621 leader_election.cc:302] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc
W20240502 20:46:54.446151 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51610: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:54.446513 11839 raft_consensus.cc:2726] T dea141ae57324a879dfd26cbeea9aead P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20240502 20:46:54.446560 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fe22224142984d69a2def286584b9aaa"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:54.446647 9687 raft_consensus.cc:482] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.446694 9687 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.447417 9687 raft_consensus.cc:504] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.447706 9142 catalog_manager.cc:5484] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 7 to 9, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 9 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:54.447969 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe22224142984d69a2def286584b9aaa" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 10 candidate_status { last_received { term: 2 index: 18 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.447969 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe22224142984d69a2def286584b9aaa" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 10 candidate_status { last_received { term: 2 index: 18 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.448058 9420 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.448058 9286 raft_consensus.cc:3032] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 LEADER]: Stepping down as leader of term 9
I20240502 20:46:54.448093 9286 raft_consensus.cc:729] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.448225 9286 consensus_queue.cc:260] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 9.21, Last appended by leader: 21, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.448357 9286 raft_consensus.cc:3037] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.448843 9420 raft_consensus.cc:2396] T fe22224142984d69a2def286584b9aaa P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 10 because replica has last-logged OpId of term: 8 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 18.
I20240502 20:46:54.449131 9286 raft_consensus.cc:2396] T fe22224142984d69a2def286584b9aaa P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 10 because replica has last-logged OpId of term: 9 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 18.
I20240502 20:46:54.449151 9687 leader_election.cc:288] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 10 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:54.449741 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:54.449831 9687 raft_consensus.cc:482] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.449877 9687 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.450628 9621 leader_election.cc:302] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:54.450829 11758 raft_consensus.cc:2726] T fe22224142984d69a2def286584b9aaa P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20240502 20:46:54.450867 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55548: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:54.450949 9687 raft_consensus.cc:504] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.451439 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 9 candidate_status { last_received { term: 6 index: 12 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
W20240502 20:46:54.451503 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:54.451529 9420 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.452153 9420 raft_consensus.cc:2454] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 9.
I20240502 20:46:54.452407 9622 leader_election.cc:302] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:54.452593 11758 raft_consensus.cc:2781] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Leader election won for term 9
I20240502 20:46:54.452661 11758 raft_consensus.cc:686] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 9 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.452647 9550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 9 candidate_status { last_received { term: 6 index: 12 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.452736 9550 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.452760 11758 consensus_queue.cc:237] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 6.12, Last appended by leader: 12, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.453508 9550 raft_consensus.cc:2396] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 9 because replica has last-logged OpId of term: 7 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 12.
I20240502 20:46:54.453671 9687 leader_election.cc:288] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 9 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:54.454416 9139 catalog_manager.cc:5484] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 7 to 9, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 9 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:54.454777 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "edebc185b24441da9504c78394378f4d"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:54.454883 9687 raft_consensus.cc:482] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.454941 9687 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.455665 9687 raft_consensus.cc:504] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
W20240502 20:46:54.455948 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51626: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:54.456004 9687 leader_election.cc:288] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 10 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:54.456629 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:54.456720 9687 raft_consensus.cc:482] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.456775 9687 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.457319 9687 raft_consensus.cc:504] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.457602 9687 leader_election.cc:288] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 10 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:54.457845 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 10 candidate_status { last_received { term: 8 index: 22 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.457935 9286 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.457918 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8d94e044dd4480a7948bc3d4aca7e2" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 10 candidate_status { last_received { term: 8 index: 22 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.458002 9424 raft_consensus.cc:3037] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.458498 9286 raft_consensus.cc:2454] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 10.
I20240502 20:46:54.458729 9621 leader_election.cc:302] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:54.458739 9424 raft_consensus.cc:2454] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 10.
I20240502 20:46:54.458868 11839 raft_consensus.cc:2781] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Leader election won for term 10
I20240502 20:46:54.458935 11839 raft_consensus.cc:686] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [term 10 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.459035 11839 consensus_queue.cc:237] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 8.22, Last appended by leader: 22, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.459739 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:54.459834 9687 raft_consensus.cc:482] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.459869 9687 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.460378 9687 raft_consensus.cc:504] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.460549 9139 catalog_manager.cc:5484] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 8 to 10, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 10 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:54.460634 9687 leader_election.cc:288] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 9 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:54.461891 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 9 candidate_status { last_received { term: 6 index: 17 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.461984 9555 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.462213 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 9 candidate_status { last_received { term: 6 index: 17 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.462306 9286 raft_consensus.cc:3032] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 LEADER]: Stepping down as leader of term 8
I20240502 20:46:54.462344 9286 raft_consensus.cc:729] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.462509 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:54.462493 9286 consensus_queue.cc:260] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 8.18, Last appended by leader: 18, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.462591 9687 raft_consensus.cc:482] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.462620 9286 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.462635 9687 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.462841 9555 raft_consensus.cc:2454] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 9.
I20240502 20:46:54.463157 9623 leader_election.cc:302] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters:
W20240502 20:46:54.463168 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59728: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:54.463378 9286 raft_consensus.cc:2396] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 9 because replica has last-logged OpId of term: 8 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 17.
I20240502 20:46:54.463336 9687 raft_consensus.cc:504] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.463598 11839 raft_consensus.cc:2781] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Leader election won for term 9
I20240502 20:46:54.463632 9687 leader_election.cc:288] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 9 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.463661 9550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edebc185b24441da9504c78394378f4d" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 10 candidate_status { last_received { term: 7 index: 14 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.463703 11839 raft_consensus.cc:686] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 9 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.463742 9550 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.463833 11839 consensus_queue.cc:237] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 6.17, Last appended by leader: 17, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.463987 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 9 candidate_status { last_received { term: 8 index: 17 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.464063 9286 raft_consensus.cc:3032] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 LEADER]: Stepping down as leader of term 8
I20240502 20:46:54.464097 9286 raft_consensus.cc:729] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.464206 9286 consensus_queue.cc:260] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 8.17, Last appended by leader: 17, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.464354 9286 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.464529 9550 raft_consensus.cc:2396] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 10 because replica has last-logged OpId of term: 8 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 14.
I20240502 20:46:54.464879 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edebc185b24441da9504c78394378f4d" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 10 candidate_status { last_received { term: 7 index: 14 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.464973 9420 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.465339 9286 raft_consensus.cc:2454] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 9.
I20240502 20:46:54.465626 9621 leader_election.cc:302] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448; no voters:
I20240502 20:46:54.465724 9420 raft_consensus.cc:2454] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 10.
I20240502 20:46:54.465824 11758 raft_consensus.cc:2781] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Leader election won for term 9
I20240502 20:46:54.465961 11758 raft_consensus.cc:686] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 9 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.466061 9622 leader_election.cc:302] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e
I20240502 20:46:54.466323 11839 raft_consensus.cc:2781] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Leader election won for term 10
I20240502 20:46:54.466420 11839 raft_consensus.cc:686] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 10 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.466521 9287 raft_consensus.cc:1261] T cb8d94e044dd4480a7948bc3d4aca7e2 P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 8 index: 21. Preceding OpId from leader: term: 10 index: 24. (index mismatch)
I20240502 20:46:54.466547 11839 consensus_queue.cc:237] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 7.14, Last appended by leader: 14, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
W20240502 20:46:54.466921 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59834: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:54.467110 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 9 candidate_status { last_received { term: 8 index: 17 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.466068 11758 consensus_queue.cc:237] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 8.17, Last appended by leader: 17, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.467278 9139 catalog_manager.cc:5484] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 8 to 9, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 9 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:54.467304 9555 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.468209 9555 raft_consensus.cc:2454] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 9.
I20240502 20:46:54.468231 9420 raft_consensus.cc:1261] T cb8d94e044dd4480a7948bc3d4aca7e2 P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 8 index: 22. Preceding OpId from leader: term: 10 index: 24. (index mismatch)
I20240502 20:46:54.468386 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d92bfee8fabd46a28ffcb7e319879406"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:54.468482 9687 raft_consensus.cc:482] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.468546 9687 raft_consensus.cc:3037] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.468564 11839 consensus_queue.cc:1035] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 20, Time since last communication: 0.000s
I20240502 20:46:54.469084 11839 consensus_queue.cc:1035] T cb8d94e044dd4480a7948bc3d4aca7e2 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20240502 20:46:54.469228 9687 raft_consensus.cc:504] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.470289 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d92bfee8fabd46a28ffcb7e319879406" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 9 candidate_status { last_received { term: 7 index: 26 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.470379 9286 raft_consensus.cc:3032] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 LEADER]: Stepping down as leader of term 8
I20240502 20:46:54.470412 9286 raft_consensus.cc:729] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.470585 9286 consensus_queue.cc:260] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 8.27, Last appended by leader: 27, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.470717 9286 raft_consensus.cc:3037] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.471474 9286 raft_consensus.cc:2396] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 9 because replica has last-logged OpId of term: 8 index: 27, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 26.
I20240502 20:46:54.471616 9687 leader_election.cc:288] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 9 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.472375 11564 mvcc.cc:205] Tried to move back new op lower bound from 7023340808050843648 to 7023340808025497600. Current Snapshot: MvccSnapshot[applied={T|T < 7023340808050843648 or (T in {7023340808050843648})}]
I20240502 20:46:54.472735 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d92bfee8fabd46a28ffcb7e319879406" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 9 candidate_status { last_received { term: 7 index: 26 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.472848 9420 raft_consensus.cc:3037] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.473683 9420 raft_consensus.cc:2454] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 9.
I20240502 20:46:54.474006 9622 leader_election.cc:302] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:54.474570 11624 raft_consensus.cc:2781] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Leader election won for term 9
I20240502 20:46:54.474643 11624 raft_consensus.cc:686] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [term 9 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.474738 11624 consensus_queue.cc:237] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 7.26, Last appended by leader: 26, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.475354 9139 catalog_manager.cc:5484] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 8 to 9, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 9 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:54.475544 9139 catalog_manager.cc:5484] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 8 to 10, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 10 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:54.476584 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:54.477069 9687 raft_consensus.cc:482] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.477121 9687 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.477814 9687 raft_consensus.cc:504] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.478330 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 10 candidate_status { last_received { term: 4 index: 12 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.478427 9423 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.478516 9139 catalog_manager.cc:5484] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 8 to 9, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 9 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:54.479249 9423 raft_consensus.cc:2396] T 9db841835d2e4b50b149c0980e762fac P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 10 because replica has last-logged OpId of term: 8 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 12.
W20240502 20:46:54.480902 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55522: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:54.481014 9687 leader_election.cc:288] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 10 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.481356 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9db841835d2e4b50b149c0980e762fac" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 10 candidate_status { last_received { term: 4 index: 12 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.481454 9287 raft_consensus.cc:3032] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 LEADER]: Stepping down as leader of term 9
I20240502 20:46:54.481493 9287 raft_consensus.cc:729] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.481606 9287 consensus_queue.cc:260] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 9.14, Last appended by leader: 14, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.481711 9287 raft_consensus.cc:3037] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.482558 9287 raft_consensus.cc:2396] T 9db841835d2e4b50b149c0980e762fac P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 10 because replica has last-logged OpId of term: 9 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 12.
I20240502 20:46:54.483377 9621 leader_election.cc:302] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:54.483403 9686 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:54.483501 9686 raft_consensus.cc:482] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.483527 11926 raft_consensus.cc:2726] T 9db841835d2e4b50b149c0980e762fac P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20240502 20:46:54.483548 9686 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 8 FOLLOWER]: Advancing to term 9
W20240502 20:46:54.483999 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55548: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:54.484349 9686 raft_consensus.cc:504] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.484680 9686 leader_election.cc:288] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 9 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
W20240502 20:46:54.485441 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:54.485801 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:54.485896 9687 raft_consensus.cc:482] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.486016 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 9 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.486089 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 9 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.486132 9424 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.486181 9555 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 8 FOLLOWER]: Advancing to term 9
I20240502 20:46:54.486781 9424 raft_consensus.cc:2454] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 9.
I20240502 20:46:54.486877 9555 raft_consensus.cc:2396] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 9 because replica has last-logged OpId of term: 7 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 14.
I20240502 20:46:54.486934 9687 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.487350 9622 leader_election.cc:302] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e
I20240502 20:46:54.487483 11839 raft_consensus.cc:2781] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Leader election won for term 9
I20240502 20:46:54.487553 11839 raft_consensus.cc:686] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 9 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.487615 9687 raft_consensus.cc:504] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.487638 11839 consensus_queue.cc:237] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 6.14, Last appended by leader: 14, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.488190 9687 leader_election.cc:288] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 10 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:54.488479 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.488571 9424 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.488762 9287 raft_consensus.cc:1261] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 8 index: 27. Preceding OpId from leader: term: 9 index: 28. (index mismatch)
I20240502 20:46:54.489149 11926 consensus_queue.cc:1035] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20240502 20:46:54.489255 9424 raft_consensus.cc:2454] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 10.
I20240502 20:46:54.489253 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.489300 11926 consensus_queue.cc:1225] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Peer 4b3cb6ad00954dc5a1270dc392870b4f log is divergent from this leader: its last log entry 8.27 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 26
I20240502 20:46:54.489349 9555 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.489516 9423 raft_consensus.cc:1261] T d92bfee8fabd46a28ffcb7e319879406 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 7 index: 26. Preceding OpId from leader: term: 9 index: 28. (index mismatch)
I20240502 20:46:54.489616 9142 catalog_manager.cc:5484] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 7 to 9, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 9 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:54.489956 9287 pending_rounds.cc:76] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f: Aborting all ops after (but not including) 26
I20240502 20:46:54.489949 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:54.489993 9287 pending_rounds.cc:97] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f: Aborting uncommitted NO_OP operation due to leader change: 8.27
I20240502 20:46:54.490021 9287 raft_consensus.cc:2866] T d92bfee8fabd46a28ffcb7e319879406 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 20:46:54.490039 9687 raft_consensus.cc:482] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.490085 9687 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.490170 9555 raft_consensus.cc:2396] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20240502 20:46:54.490789 9687 raft_consensus.cc:504] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.490813 9622 leader_election.cc:302] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:54.491091 9687 leader_election.cc:288] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 10 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.491333 11624 raft_consensus.cc:2781] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Leader election won for term 10
I20240502 20:46:54.491415 11624 raft_consensus.cc:686] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 10 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.491524 11624 consensus_queue.cc:237] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.491691 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 10 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.491788 9287 raft_consensus.cc:3032] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 LEADER]: Stepping down as leader of term 9
I20240502 20:46:54.491823 9287 raft_consensus.cc:729] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.491938 9287 consensus_queue.cc:260] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 9.17, Last appended by leader: 17, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.491961 11624 consensus_queue.cc:1035] T d92bfee8fabd46a28ffcb7e319879406 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20240502 20:46:54.492075 9287 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.492906 9287 raft_consensus.cc:2396] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 10 because replica has last-logged OpId of term: 9 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 14.
I20240502 20:46:54.493902 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 10 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.493991 9555 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.494855 9555 raft_consensus.cc:2396] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 10 because replica has last-logged OpId of term: 8 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 14.
I20240502 20:46:54.495296 9139 catalog_manager.cc:5484] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 8 to 10, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 10 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:54.495848 9623 leader_election.cc:302] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f
I20240502 20:46:54.496009 11926 raft_consensus.cc:2726] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20240502 20:46:54.496183 9517 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51626: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
W20240502 20:46:54.496215 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51610: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:54.496747 9687 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:54.496887 9687 raft_consensus.cc:482] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.496940 9687 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.497670 9687 raft_consensus.cc:504] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.498181 9687 leader_election.cc:288] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 10 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.498236 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 10 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.498318 9287 raft_consensus.cc:3032] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 LEADER]: Stepping down as leader of term 9
I20240502 20:46:54.498353 9287 raft_consensus.cc:729] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.498502 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd36ed4462ca494c87169bd2ca7add45" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 10 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.498495 9287 consensus_queue.cc:260] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.12, Last appended by leader: 12, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.498596 9423 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.498617 9287 raft_consensus.cc:3037] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.499056 9686 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:54.499148 9686 raft_consensus.cc:482] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.499197 9686 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.499446 9287 raft_consensus.cc:2396] T cd36ed4462ca494c87169bd2ca7add45 P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 10 because replica has last-logged OpId of term: 9 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 9.
I20240502 20:46:54.499454 9423 raft_consensus.cc:2396] T cd36ed4462ca494c87169bd2ca7add45 P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 10 because replica has last-logged OpId of term: 8 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 9.
I20240502 20:46:54.499955 9686 raft_consensus.cc:504] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.500277 9686 leader_election.cc:288] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 10 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:54.500516 9621 leader_election.cc:302] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:54.500723 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 10 candidate_status { last_received { term: 9 index: 24 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.500828 9423 raft_consensus.cc:3032] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 LEADER]: Stepping down as leader of term 9
I20240502 20:46:54.500867 9423 raft_consensus.cc:729] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.500980 9423 consensus_queue.cc:260] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 9.24, Last appended by leader: 24, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.501041 9550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 10 candidate_status { last_received { term: 9 index: 24 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.501109 9423 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.501719 11758 raft_consensus.cc:2726] T cd36ed4462ca494c87169bd2ca7add45 P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20240502 20:46:54.501775 9423 raft_consensus.cc:2454] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 10.
I20240502 20:46:54.501945 9686 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e"
dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
from {username='slave'} at 127.0.0.1:40128
I20240502 20:46:54.502040 9686 raft_consensus.cc:482] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.502080 9622 leader_election.cc:302] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e
I20240502 20:46:54.502089 9686 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.502841 11926 raft_consensus.cc:2781] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Leader election won for term 10
I20240502 20:46:54.503113 9686 raft_consensus.cc:504] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.503147 11926 raft_consensus.cc:686] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 10 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.503289 11926 consensus_queue.cc:237] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 9.24, Last appended by leader: 24, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.503443 9686 leader_election.cc:288] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 10 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:54.503723 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.503822 9424 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.503890 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e" candidate_uuid: "64136bd82a30486e9d56fc28da4bd448" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.503968 9555 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.504571 9424 raft_consensus.cc:2454] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64136bd82a30486e9d56fc28da4bd448 in term 10.
I20240502 20:46:54.504680 9555 raft_consensus.cc:2396] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64136bd82a30486e9d56fc28da4bd448 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20240502 20:46:54.504915 9622 leader_election.cc:302] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
W20240502 20:46:54.505299 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59834: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:54.505539 9142 catalog_manager.cc:5484] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 9 to 10, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 10 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:54.506114 11926 raft_consensus.cc:2781] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Leader election won for term 10
I20240502 20:46:54.506290 11926 raft_consensus.cc:686] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 10 LEADER]: Becoming Leader. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.506520 11926 consensus_queue.cc:237] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.509768 9139 catalog_manager.cc:5484] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 reported cstate change: term changed from 8 to 10, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68). New cstate: current_term: 10 leader_uuid: "64136bd82a30486e9d56fc28da4bd448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
W20240502 20:46:54.511545 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59728: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
W20240502 20:46:54.513602 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55520: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:54.516964 9423 raft_consensus.cc:1261] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 9 index: 24. Preceding OpId from leader: term: 10 index: 26. (index mismatch)
I20240502 20:46:54.517041 9555 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.517139 9555 raft_consensus.cc:482] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.517194 9555 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.517349 11839 consensus_queue.cc:1035] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.000s
I20240502 20:46:54.517524 9550 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.517951 9555 raft_consensus.cc:504] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.518317 9555 leader_election.cc:288] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 10 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:54.518358 9550 raft_consensus.cc:1261] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Refusing update from remote peer 64136bd82a30486e9d56fc28da4bd448: Log matching property violated. Preceding OpId in replica: term: 9 index: 24. Preceding OpId from leader: term: 10 index: 26. (index mismatch)
I20240502 20:46:54.518596 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.518689 9686 raft_consensus.cc:3032] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 9 LEADER]: Stepping down as leader of term 9
I20240502 20:46:54.518723 9686 raft_consensus.cc:729] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.518831 9686 consensus_queue.cc:260] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.518936 9686 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.519709 9686 raft_consensus.cc:2396] T c84bf8e0914f4ed4806e1e52d37c891a P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20240502 20:46:54.519965 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84bf8e0914f4ed4806e1e52d37c891a" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.520057 9424 raft_consensus.cc:3037] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.520857 9424 raft_consensus.cc:2454] T c84bf8e0914f4ed4806e1e52d37c891a P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 10.
I20240502 20:46:54.521772 11839 consensus_queue.cc:1035] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 23, Time since last communication: 0.000s
I20240502 20:46:54.522050 9491 leader_election.cc:302] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:54.522209 11577 raft_consensus.cc:2781] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Leader election won for term 10
I20240502 20:46:54.522300 11577 raft_consensus.cc:686] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.522416 11577 consensus_queue.cc:237] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.522760 9550 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.522858 9550 raft_consensus.cc:482] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.522907 9550 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.523651 9550 raft_consensus.cc:504] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.523968 9550 leader_election.cc:288] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.524322 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.524417 9686 raft_consensus.cc:3032] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 10 LEADER]: Stepping down as leader of term 10
W20240502 20:46:54.524431 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55548: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:54.524454 9686 raft_consensus.cc:729] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.524502 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6278a8937308475d88b2b53e634dcdd5" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.524590 9287 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.524564 9686 consensus_queue.cc:260] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.524682 9686 raft_consensus.cc:3037] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.525511 9686 raft_consensus.cc:2396] T 6278a8937308475d88b2b53e634dcdd5 P 64136bd82a30486e9d56fc28da4bd448 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20240502 20:46:54.525621 9287 raft_consensus.cc:2396] T 6278a8937308475d88b2b53e634dcdd5 P 4b3cb6ad00954dc5a1270dc392870b4f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20240502 20:46:54.525619 9142 catalog_manager.cc:5484] T c84bf8e0914f4ed4806e1e52d37c891a P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 9 to 10, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 10 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:54.526029 9492 leader_election.cc:302] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:54.526158 9555 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.526245 9555 raft_consensus.cc:482] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.526311 9555 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.527004 9555 raft_consensus.cc:504] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.527314 9555 leader_election.cc:288] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.527813 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.527896 9287 raft_consensus.cc:3032] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 LEADER]: Stepping down as leader of term 10
I20240502 20:46:54.527897 11578 raft_consensus.cc:2726] T 6278a8937308475d88b2b53e634dcdd5 P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20240502 20:46:54.527925 9287 raft_consensus.cc:729] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.527949 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.528033 9424 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.528061 9287 consensus_queue.cc:260] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.528266 9287 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.528831 9424 raft_consensus.cc:2396] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 20:46:54.529022 9555 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.529120 9555 raft_consensus.cc:482] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.529170 9555 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.531230 9287 raft_consensus.cc:2396] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 20:46:54.531850 9490 leader_election.cc:302] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:54.532466 11577 raft_consensus.cc:2726] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20240502 20:46:54.533175 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55522: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:54.533237 9555 raft_consensus.cc:504] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.533958 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 40 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.534057 9686 raft_consensus.cc:3032] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 10 LEADER]: Stepping down as leader of term 10
I20240502 20:46:54.534094 9686 raft_consensus.cc:729] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.534204 9686 consensus_queue.cc:260] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 10.41, Last appended by leader: 41, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.534305 9686 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.534744 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 40 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.534840 9424 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.535111 9686 raft_consensus.cc:2396] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 11 because replica has last-logged OpId of term: 10 index: 41, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 40.
I20240502 20:46:54.535286 9555 leader_election.cc:288] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.535609 9424 raft_consensus.cc:2454] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 11.
I20240502 20:46:54.535940 9491 leader_election.cc:302] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:54.536401 11577 raft_consensus.cc:2781] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Leader election won for term 11
I20240502 20:46:54.536484 11577 raft_consensus.cc:686] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.536589 11577 consensus_queue.cc:237] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 9.40, Last appended by leader: 40, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.537117 9550 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.537222 9550 raft_consensus.cc:482] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.537276 9550 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Advancing to term 11
W20240502 20:46:54.537923 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51626: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:54.538013 9550 raft_consensus.cc:504] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.538161 9139 catalog_manager.cc:5484] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 10 to 11, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 11 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:54.538667 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 33 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.538667 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6aa5b2d4c04d2bb971dabc21a4544a" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 33 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.538761 9287 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.538761 9686 raft_consensus.cc:3032] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 10 LEADER]: Stepping down as leader of term 10
I20240502 20:46:54.538806 9686 raft_consensus.cc:729] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.538918 9686 consensus_queue.cc:260] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 10.34, Last appended by leader: 34, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.539029 9686 raft_consensus.cc:3037] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.539584 9287 raft_consensus.cc:2396] T bf6aa5b2d4c04d2bb971dabc21a4544a P 4b3cb6ad00954dc5a1270dc392870b4f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 11 because replica has last-logged OpId of term: 9 index: 34, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 33.
I20240502 20:46:54.539805 9686 raft_consensus.cc:2396] T bf6aa5b2d4c04d2bb971dabc21a4544a P 64136bd82a30486e9d56fc28da4bd448 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 11 because replica has last-logged OpId of term: 10 index: 34, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 33.
I20240502 20:46:54.540184 9490 leader_election.cc:302] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:54.540324 11577 raft_consensus.cc:2726] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20240502 20:46:54.540397 9550 leader_election.cc:288] T bf6aa5b2d4c04d2bb971dabc21a4544a P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.542016 9550 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.542117 9550 raft_consensus.cc:482] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.542176 9550 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.542870 9550 raft_consensus.cc:504] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.543164 9550 leader_election.cc:288] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 10 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.543488 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 43 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.543578 9686 raft_consensus.cc:3032] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 9 LEADER]: Stepping down as leader of term 9
I20240502 20:46:54.543615 9686 raft_consensus.cc:729] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.543721 9686 consensus_queue.cc:260] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 43, Committed index: 43, Last appended: 9.44, Last appended by leader: 44, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.543845 9686 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.544588 9686 raft_consensus.cc:2396] T b6842838ea9b47d7bd7c343a16ee8768 P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 10 because replica has last-logged OpId of term: 9 index: 44, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 43.
I20240502 20:46:54.545212 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6842838ea9b47d7bd7c343a16ee8768" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 43 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.545308 9286 raft_consensus.cc:3037] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.546152 9286 raft_consensus.cc:2454] T b6842838ea9b47d7bd7c343a16ee8768 P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 10.
I20240502 20:46:54.546553 9490 leader_election.cc:302] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters: 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:54.546756 11577 raft_consensus.cc:2781] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Leader election won for term 10
I20240502 20:46:54.546840 11577 raft_consensus.cc:686] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.546944 11577 consensus_queue.cc:237] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 43, Committed index: 43, Last appended: 8.43, Last appended by leader: 43, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.547636 9550 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.547725 9550 raft_consensus.cc:482] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.547783 9550 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.548494 9550 raft_consensus.cc:504] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.548802 9550 leader_election.cc:288] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.549284 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.549379 9286 raft_consensus.cc:3032] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 LEADER]: Stepping down as leader of term 10
I20240502 20:46:54.549415 9286 raft_consensus.cc:729] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.549535 9286 consensus_queue.cc:260] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.549669 9286 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.549710 9139 catalog_manager.cc:5484] T b6842838ea9b47d7bd7c343a16ee8768 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 9 to 10, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 10 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:54.550434 9286 raft_consensus.cc:2396] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20240502 20:46:54.550884 9550 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3315525f1849462fbf49a285f424ea51"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
W20240502 20:46:54.550901 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59834: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:54.550984 9550 raft_consensus.cc:482] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.551043 9550 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.551775 9550 raft_consensus.cc:504] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.552210 9550 leader_election.cc:288] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.552366 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3315525f1849462fbf49a285f424ea51" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 41 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.552457 9286 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.553246 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3315525f1849462fbf49a285f424ea51" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 41 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.553277 9286 raft_consensus.cc:2454] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 11.
W20240502 20:46:54.553325 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51610: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:54.553344 9420 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.553519 9490 leader_election.cc:302] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:54.553660 11578 raft_consensus.cc:2781] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Leader election won for term 11
I20240502 20:46:54.553726 11578 raft_consensus.cc:686] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.553843 11578 consensus_queue.cc:237] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 8.41, Last appended by leader: 41, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.554262 9420 raft_consensus.cc:2396] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 11 because replica has last-logged OpId of term: 9 index: 41, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 41.
I20240502 20:46:54.554450 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.554576 9423 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.555402 9423 raft_consensus.cc:2396] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20240502 20:46:54.555406 9550 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.555503 9550 raft_consensus.cc:482] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.555558 9550 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.556066 9139 catalog_manager.cc:5484] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 9 to 11, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 11 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:54.556275 9550 raft_consensus.cc:504] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.556574 9550 leader_election.cc:288] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.557664 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 13 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.557741 9491 leader_election.cc:302] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:54.557760 9423 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.558534 11577 raft_consensus.cc:2726] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20240502 20:46:54.559953 9423 raft_consensus.cc:2396] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 11 because replica has last-logged OpId of term: 9 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 13.
I20240502 20:46:54.560441 9553 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.560545 9553 raft_consensus.cc:482] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.560603 9553 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.561389 9553 raft_consensus.cc:504] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.561694 9553 leader_election.cc:288] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 10 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:54.562136 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546ac55ccb0b4e91a17074a8ee8e99ef" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 13 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.562238 9286 raft_consensus.cc:3037] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.562379 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 14 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.562492 9686 raft_consensus.cc:3032] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 9 LEADER]: Stepping down as leader of term 9
I20240502 20:46:54.562531 9686 raft_consensus.cc:729] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.562641 9686 consensus_queue.cc:260] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 9.15, Last appended by leader: 15, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.562773 9686 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.563043 9286 raft_consensus.cc:2454] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 11.
I20240502 20:46:54.563462 9490 leader_election.cc:302] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters: 88a19090795d47c2a1ea1dd979fb62bc
I20240502 20:46:54.563602 9686 raft_consensus.cc:2396] T d25d4ef404e34234a1a50da88deb5f90 P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 10 because replica has last-logged OpId of term: 9 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 14.
I20240502 20:46:54.563661 11577 raft_consensus.cc:2781] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Leader election won for term 11
I20240502 20:46:54.563732 11577 raft_consensus.cc:686] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.563844 11577 consensus_queue.cc:237] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 8.13, Last appended by leader: 13, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.564365 9553 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "edebc185b24441da9504c78394378f4d"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.564460 9553 raft_consensus.cc:482] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.564508 9553 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.565255 9553 raft_consensus.cc:504] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.565558 9553 leader_election.cc:288] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:54.565928 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edebc185b24441da9504c78394378f4d" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 14 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.566020 9686 raft_consensus.cc:3032] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 10 LEADER]: Stepping down as leader of term 10
I20240502 20:46:54.566061 9686 raft_consensus.cc:729] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.566159 9686 consensus_queue.cc:260] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 10.15, Last appended by leader: 15, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.566273 9686 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.567168 9686 raft_consensus.cc:2396] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 11 because replica has last-logged OpId of term: 10 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 14.
W20240502 20:46:54.567296 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55548: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:54.567790 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25d4ef404e34234a1a50da88deb5f90" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 14 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.567878 9423 raft_consensus.cc:3037] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.568226 9144 catalog_manager.cc:5484] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 9 to 11, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 11 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:54.568820 9423 raft_consensus.cc:2454] T d25d4ef404e34234a1a50da88deb5f90 P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 10.
I20240502 20:46:54.569428 9491 leader_election.cc:302] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:54.569546 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edebc185b24441da9504c78394378f4d" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 14 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.569631 9420 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.569675 11578 raft_consensus.cc:2781] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Leader election won for term 10
I20240502 20:46:54.569743 11578 raft_consensus.cc:686] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.569855 11578 consensus_queue.cc:237] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 7.14, Last appended by leader: 14, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.570488 9420 raft_consensus.cc:2454] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 11.
I20240502 20:46:54.571319 9553 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.571344 9491 leader_election.cc:302] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:54.571420 9553 raft_consensus.cc:482] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.571471 9553 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.572224 9553 raft_consensus.cc:504] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
W20240502 20:46:54.572450 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59728: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:54.572564 9553 leader_election.cc:288] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 10 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.571477 11577 raft_consensus.cc:2781] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Leader election won for term 11
I20240502 20:46:54.572736 11577 raft_consensus.cc:686] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.572839 11577 consensus_queue.cc:237] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 8.14, Last appended by leader: 14, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.572889 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 17 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.572976 9686 raft_consensus.cc:3032] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 9 LEADER]: Stepping down as leader of term 9
I20240502 20:46:54.573010 9686 raft_consensus.cc:729] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.573247 9686 consensus_queue.cc:260] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 9.18, Last appended by leader: 18, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.573385 9686 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.574151 9686 raft_consensus.cc:2396] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 10 because replica has last-logged OpId of term: 9 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 17.
I20240502 20:46:54.575265 9144 catalog_manager.cc:5484] T d25d4ef404e34234a1a50da88deb5f90 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 9 to 10, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 10 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:54.575984 9553 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.576347 9553 raft_consensus.cc:482] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.576402 9553 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.576647 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fddd0fdac3ae414a8b2610752e96864b" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 17 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.576735 9286 raft_consensus.cc:3037] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.577107 9553 raft_consensus.cc:504] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.578002 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 12 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.578100 9420 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.578197 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b6c9892e6c49b2a424aa06cdf3dfd9" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 12 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.578284 9686 raft_consensus.cc:3032] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 9 LEADER]: Stepping down as leader of term 9
I20240502 20:46:54.578318 9686 raft_consensus.cc:729] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.578424 9686 consensus_queue.cc:260] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.13, Last appended by leader: 13, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.578577 9686 raft_consensus.cc:3037] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.578929 9420 raft_consensus.cc:2454] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 10.
I20240502 20:46:54.579267 9686 raft_consensus.cc:2396] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 10 because replica has last-logged OpId of term: 9 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 12.
I20240502 20:46:54.579488 9491 leader_election.cc:302] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:54.579624 11578 raft_consensus.cc:2781] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Leader election won for term 10
I20240502 20:46:54.579690 11578 raft_consensus.cc:686] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.579792 11578 consensus_queue.cc:237] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 7.12, Last appended by leader: 12, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.579869 9553 leader_election.cc:288] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 10 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.580049 9144 catalog_manager.cc:5484] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 10 to 11, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 11 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:54.580899 9286 raft_consensus.cc:2454] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 10.
I20240502 20:46:54.582283 9490 leader_election.cc:302] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters: 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:54.582429 11577 raft_consensus.cc:2781] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Leader election won for term 10
I20240502 20:46:54.582526 11577 raft_consensus.cc:686] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.582630 11577 consensus_queue.cc:237] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 8.17, Last appended by leader: 17, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.583225 9553 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.583359 9553 raft_consensus.cc:482] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.583437 9553 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.584194 9553 raft_consensus.cc:504] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.584538 9553 leader_election.cc:288] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.584941 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 42 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.585041 9287 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.585920 9287 raft_consensus.cc:2396] T acd3076bc7344e7694362ea9ffa35e54 P 4b3cb6ad00954dc5a1270dc392870b4f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 11 because replica has last-logged OpId of term: 9 index: 43, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 42.
I20240502 20:46:54.585980 9289 raft_consensus.cc:1261] T 546ac55ccb0b4e91a17074a8ee8e99ef P 4b3cb6ad00954dc5a1270dc392870b4f [term 11 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 7 index: 12. Preceding OpId from leader: term: 11 index: 15. (index mismatch)
I20240502 20:46:54.586284 9144 catalog_manager.cc:5484] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 9 to 10, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 10 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:54.586524 9420 raft_consensus.cc:1261] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 11 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 9 index: 13. Preceding OpId from leader: term: 11 index: 15. (index mismatch)
I20240502 20:46:54.586637 11577 consensus_queue.cc:1035] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 20:46:54.588165 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acd3076bc7344e7694362ea9ffa35e54" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 42 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.588274 9686 raft_consensus.cc:3032] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 10 LEADER]: Stepping down as leader of term 10
I20240502 20:46:54.588323 9686 raft_consensus.cc:729] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.588429 9686 consensus_queue.cc:260] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 42, Committed index: 42, Last appended: 10.43, Last appended by leader: 43, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.588552 9686 raft_consensus.cc:3037] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.590001 11577 consensus_queue.cc:1035] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 20:46:54.590071 9550 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "20698b2531674eefb9df3125a86653ba"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.590103 11577 consensus_queue.cc:1225] T 546ac55ccb0b4e91a17074a8ee8e99ef P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Peer 88a19090795d47c2a1ea1dd979fb62bc log is divergent from this leader: its last log entry 9.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20240502 20:46:54.590281 9550 raft_consensus.cc:482] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.590442 9550 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.591002 9686 raft_consensus.cc:2396] T acd3076bc7344e7694362ea9ffa35e54 P 64136bd82a30486e9d56fc28da4bd448 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 11 because replica has last-logged OpId of term: 10 index: 43, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 42.
I20240502 20:46:54.591311 9492 leader_election.cc:302] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:54.591513 11577 raft_consensus.cc:2726] T acd3076bc7344e7694362ea9ffa35e54 P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20240502 20:46:54.591317 9550 raft_consensus.cc:504] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.592497 9550 leader_election.cc:288] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.592733 9139 catalog_manager.cc:5484] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 9 to 10, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 10 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:54.593415 11536 mvcc.cc:205] Tried to move back new op lower bound from 7023340808538791936 to 7023340808454774784. Current Snapshot: MvccSnapshot[applied={T|T < 7023340808538791936}]
I20240502 20:46:54.594121 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20698b2531674eefb9df3125a86653ba" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 17 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.594228 9286 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.595077 9286 raft_consensus.cc:2454] T 20698b2531674eefb9df3125a86653ba P 4b3cb6ad00954dc5a1270dc392870b4f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 11.
W20240502 20:46:54.596295 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40188: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:54.596937 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20698b2531674eefb9df3125a86653ba" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 17 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.597035 9424 raft_consensus.cc:3037] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.597931 9424 raft_consensus.cc:2396] T 20698b2531674eefb9df3125a86653ba P 88a19090795d47c2a1ea1dd979fb62bc [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 11 because replica has last-logged OpId of term: 9 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 17.
I20240502 20:46:54.598073 9490 leader_election.cc:302] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:54.598204 11577 raft_consensus.cc:2781] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Leader election won for term 11
I20240502 20:46:54.598280 11577 raft_consensus.cc:686] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.598294 9424 raft_consensus.cc:1261] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 11 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 9 index: 41. Preceding OpId from leader: term: 11 index: 43. (index mismatch)
I20240502 20:46:54.598387 11577 consensus_queue.cc:237] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 8.17, Last appended by leader: 17, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.598753 9423 pending_rounds.cc:76] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc: Aborting all ops after (but not including) 12
I20240502 20:46:54.598783 11578 consensus_queue.cc:1035] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 42, Last known committed idx: 40, Time since last communication: 0.000s
I20240502 20:46:54.598835 9423 pending_rounds.cc:97] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc: Aborting uncommitted NO_OP operation due to leader change: 9.13
I20240502 20:46:54.598876 11578 consensus_queue.cc:1225] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Peer 88a19090795d47c2a1ea1dd979fb62bc log is divergent from this leader: its last log entry 9.41 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 40
I20240502 20:46:54.599177 9423 raft_consensus.cc:2866] T 546ac55ccb0b4e91a17074a8ee8e99ef P 88a19090795d47c2a1ea1dd979fb62bc [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 20:46:54.599206 9286 raft_consensus.cc:1261] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 11 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 7 index: 40. Preceding OpId from leader: term: 11 index: 43. (index mismatch)
I20240502 20:46:54.599510 11578 consensus_queue.cc:1035] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 42, Last known committed idx: 40, Time since last communication: 0.000s
I20240502 20:46:54.600135 9553 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.600279 9553 raft_consensus.cc:482] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.600378 9553 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.601020 9139 catalog_manager.cc:5484] T 20698b2531674eefb9df3125a86653ba P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 9 to 11, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 11 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:54.601171 9553 raft_consensus.cc:504] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.603531 9553 leader_election.cc:288] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
W20240502 20:46:54.604239 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51610: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
W20240502 20:46:54.604761 9383 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55548: Illegal state: replica 88a19090795d47c2a1ea1dd979fb62bc is not leader of this config: current role FOLLOWER
I20240502 20:46:54.606407 9553 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.606536 9553 raft_consensus.cc:482] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.606597 9553 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.607328 9553 raft_consensus.cc:504] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
W20240502 20:46:54.608479 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59728: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:54.610410 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 14 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.610591 9289 raft_consensus.cc:3032] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 LEADER]: Stepping down as leader of term 10
I20240502 20:46:54.610641 9289 raft_consensus.cc:729] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.610723 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.610816 9424 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.610783 9289 consensus_queue.cc:260] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 10.15, Last appended by leader: 15, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.610911 9289 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.611611 9424 raft_consensus.cc:2454] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 11.
I20240502 20:46:54.611771 9289 raft_consensus.cc:2396] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 11 because replica has last-logged OpId of term: 10 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 14.
I20240502 20:46:54.612308 9553 leader_election.cc:288] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
I20240502 20:46:54.612533 9491 leader_election.cc:302] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:54.612815 9553 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.612884 11578 raft_consensus.cc:2781] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Leader election won for term 11
I20240502 20:46:54.612951 11578 raft_consensus.cc:686] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.612967 9553 raft_consensus.cc:482] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.613014 9553 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.613054 11578 consensus_queue.cc:237] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.613698 9687 raft_consensus.cc:1261] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 9 index: 13. Preceding OpId from leader: term: 10 index: 14. (index mismatch)
I20240502 20:46:54.613706 9553 raft_consensus.cc:504] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.614063 11578 consensus_queue.cc:1035] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 20:46:54.614152 11578 consensus_queue.cc:1225] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Peer 64136bd82a30486e9d56fc28da4bd448 log is divergent from this leader: its last log entry 9.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20240502 20:46:54.614691 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 17 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.614738 9687 pending_rounds.cc:76] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448: Aborting all ops after (but not including) 10
I20240502 20:46:54.614789 9687 pending_rounds.cc:97] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448: Aborting uncommitted NO_OP operation due to leader change: 3.11
I20240502 20:46:54.614799 9289 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.614817 9687 raft_consensus.cc:2866] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 20:46:54.614851 9687 pending_rounds.cc:97] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448: Aborting uncommitted NO_OP operation due to leader change: 6.12
I20240502 20:46:54.614871 9687 raft_consensus.cc:2866] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 20:46:54.614897 9687 pending_rounds.cc:97] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448: Aborting uncommitted NO_OP operation due to leader change: 9.13
I20240502 20:46:54.614915 9687 raft_consensus.cc:2866] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 20:46:54.615218 9140 catalog_manager.cc:5484] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 10 to 11, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 11 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:54.615617 9289 raft_consensus.cc:2396] T b59d99b81d3d4528a74c045f6ca41199 P 4b3cb6ad00954dc5a1270dc392870b4f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 11 because replica has last-logged OpId of term: 9 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 17.
W20240502 20:46:54.616112 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40118: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:54.616484 9553 leader_election.cc:288] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.617049 9553 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.617193 9553 raft_consensus.cc:482] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.617712 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b59d99b81d3d4528a74c045f6ca41199" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 17 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.617808 9688 raft_consensus.cc:3037] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.618338 9553 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.618692 9688 raft_consensus.cc:2454] T b59d99b81d3d4528a74c045f6ca41199 P 64136bd82a30486e9d56fc28da4bd448 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 11.
I20240502 20:46:54.619614 9492 leader_election.cc:302] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448; no voters: 4b3cb6ad00954dc5a1270dc392870b4f
I20240502 20:46:54.619812 11577 raft_consensus.cc:2781] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Leader election won for term 11
I20240502 20:46:54.619894 11577 raft_consensus.cc:686] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.620004 11577 consensus_queue.cc:237] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 8.17, Last appended by leader: 17, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.620404 9553 raft_consensus.cc:504] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.620972 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 14 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.621003 9553 leader_election.cc:288] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.621076 9420 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.621207 9687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 32 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.621311 9687 raft_consensus.cc:3032] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 10 LEADER]: Stepping down as leader of term 10
I20240502 20:46:54.621362 9687 raft_consensus.cc:729] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.621492 9687 consensus_queue.cc:260] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 10.33, Last appended by leader: 33, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.621592 9687 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.621903 9420 raft_consensus.cc:2454] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 11.
I20240502 20:46:54.622071 9553 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.622242 9420 raft_consensus.cc:1261] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 10 index: 14. (index mismatch)
I20240502 20:46:54.622378 9553 raft_consensus.cc:482] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.622535 9687 raft_consensus.cc:2396] T e553edf81f684b11aaafa549b5f8c7b7 P 64136bd82a30486e9d56fc28da4bd448 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 11 because replica has last-logged OpId of term: 10 index: 33, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 32.
I20240502 20:46:54.623834 9491 leader_election.cc:302] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 4b3cb6ad00954dc5a1270dc392870b4f
I20240502 20:46:54.623979 11577 raft_consensus.cc:2781] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Leader election won for term 11
I20240502 20:46:54.624068 11577 raft_consensus.cc:686] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.624174 9420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e553edf81f684b11aaafa549b5f8c7b7" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 32 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.624161 11578 consensus_queue.cc:1035] T 61b6c9892e6c49b2a424aa06cdf3dfd9 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 20:46:54.624274 9420 raft_consensus.cc:3037] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.623965 9553 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.624168 11577 consensus_queue.cc:237] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 8.14, Last appended by leader: 14, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.625036 9420 raft_consensus.cc:2454] T e553edf81f684b11aaafa549b5f8c7b7 P 88a19090795d47c2a1ea1dd979fb62bc [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 11.
I20240502 20:46:54.625200 9553 raft_consensus.cc:504] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.625890 9553 leader_election.cc:288] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815)
W20240502 20:46:54.626397 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40118: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:54.627043 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.627137 9688 raft_consensus.cc:3032] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 10 LEADER]: Stepping down as leader of term 10
I20240502 20:46:54.627174 9688 raft_consensus.cc:729] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.627173 11707 mvcc.cc:205] Tried to move back new op lower bound from 7023340808653225984 to 7023340803050905600. Current Snapshot: MvccSnapshot[applied={T|T < 7023340808653225984}]
I20240502 20:46:54.627312 9688 consensus_queue.cc:260] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.627434 9688 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.627790 9553 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.627883 9553 raft_consensus.cc:482] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.628248 9553 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.628276 9688 raft_consensus.cc:2396] T c17aac2ef2a74cf6a6f92f84b908f40e P 64136bd82a30486e9d56fc28da4bd448 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20240502 20:46:54.629024 9491 leader_election.cc:302] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:54.629160 11578 raft_consensus.cc:2781] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Leader election won for term 11
I20240502 20:46:54.629225 11578 raft_consensus.cc:686] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.629331 11578 consensus_queue.cc:237] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 10.32, Last appended by leader: 32, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.629776 9553 raft_consensus.cc:504] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.629848 9141 catalog_manager.cc:5484] T b59d99b81d3d4528a74c045f6ca41199 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 9 to 11, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 11 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } }
I20240502 20:46:54.630120 9553 leader_election.cc:288] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 10 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.630246 9141 catalog_manager.cc:5484] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 10 to 11, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 11 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:54.630384 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 12 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.630510 9289 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.631376 9289 raft_consensus.cc:2454] T da71e448ed304b338946568c4ee94fe6 P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 10.
I20240502 20:46:54.631618 9490 leader_election.cc:302] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 4b3cb6ad00954dc5a1270dc392870b4f; no voters:
I20240502 20:46:54.631937 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c17aac2ef2a74cf6a6f92f84b908f40e" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.632100 9424 raft_consensus.cc:3037] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.632230 11578 raft_consensus.cc:2781] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Leader election won for term 10
I20240502 20:46:54.632351 11578 raft_consensus.cc:686] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.632540 11578 consensus_queue.cc:237] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 7.12, Last appended by leader: 12, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
W20240502 20:46:54.632617 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59728: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:54.632980 9424 raft_consensus.cc:2454] T c17aac2ef2a74cf6a6f92f84b908f40e P 88a19090795d47c2a1ea1dd979fb62bc [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 11.
I20240502 20:46:54.633241 9553 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.633404 9553 raft_consensus.cc:482] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.633508 9553 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.633941 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.634044 9686 raft_consensus.cc:3032] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 10 LEADER]: Stepping down as leader of term 10
I20240502 20:46:54.634090 9686 raft_consensus.cc:729] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.634240 9686 consensus_queue.cc:260] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.634389 9686 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Advancing to term 11
I20240502 20:46:54.634542 9553 raft_consensus.cc:504] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.635133 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 10 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.635227 9289 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.635322 9686 raft_consensus.cc:2396] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20240502 20:46:54.635728 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da71e448ed304b338946568c4ee94fe6" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 12 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.635820 9421 raft_consensus.cc:3037] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.636054 9289 raft_consensus.cc:2396] T 867c2c1a772f4133a2a83e1e9ea2a831 P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 10 because replica has last-logged OpId of term: 8 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 15.
I20240502 20:46:54.636675 9421 raft_consensus.cc:2396] T da71e448ed304b338946568c4ee94fe6 P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 10 because replica has last-logged OpId of term: 8 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 12.
I20240502 20:46:54.637977 9140 catalog_manager.cc:5484] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 10 to 11, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 11 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:54.638581 9491 leader_election.cc:302] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:54.638536 9140 catalog_manager.cc:5484] T da71e448ed304b338946568c4ee94fe6 P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 8 to 10, leader changed from 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 10 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } }
I20240502 20:46:54.638772 11577 raft_consensus.cc:2781] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Leader election won for term 11
I20240502 20:46:54.638854 11577 raft_consensus.cc:686] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.638887 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867c2c1a772f4133a2a83e1e9ea2a831" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 10 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.638979 9688 raft_consensus.cc:3032] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 9 LEADER]: Stepping down as leader of term 9
I20240502 20:46:54.638967 11577 consensus_queue.cc:237] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.639019 9688 raft_consensus.cc:729] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 64136bd82a30486e9d56fc28da4bd448, State: Running, Role: LEADER
I20240502 20:46:54.639187 9688 consensus_queue.cc:260] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 9.18, Last appended by leader: 18, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.639319 9688 raft_consensus.cc:3037] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.640265 9688 raft_consensus.cc:2396] T 867c2c1a772f4133a2a83e1e9ea2a831 P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 10 because replica has last-logged OpId of term: 9 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 15.
I20240502 20:46:54.640514 9492 leader_election.cc:302] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e; no voters: 4b3cb6ad00954dc5a1270dc392870b4f, 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:54.640816 11578 raft_consensus.cc:2726] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20240502 20:46:54.641021 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40188: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:54.642323 9553 leader_election.cc:288] T 867c2c1a772f4133a2a83e1e9ea2a831 P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 10 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.642372 9423 pending_rounds.cc:76] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc: Aborting all ops after (but not including) 40
I20240502 20:46:54.642405 9423 pending_rounds.cc:97] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc: Aborting uncommitted NO_OP operation due to leader change: 9.41
I20240502 20:46:54.642432 9423 raft_consensus.cc:2866] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20240502 20:46:54.643020 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51610: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:54.644097 9553 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db"
dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
from {username='slave'} at 127.0.0.1:51608
I20240502 20:46:54.644198 9553 raft_consensus.cc:482] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.644258 9553 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.644999 9553 raft_consensus.cc:504] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.645056 9550 raft_consensus.cc:1226] T e553edf81f684b11aaafa549b5f8c7b7 P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 LEADER]: Rejecting Update request from peer 64136bd82a30486e9d56fc28da4bd448 for earlier term 10. Current term is 11. Ops: []
I20240502 20:46:54.645262 9142 catalog_manager.cc:5484] T c17aac2ef2a74cf6a6f92f84b908f40e P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 10 to 11, leader changed from 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 11 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } }
I20240502 20:46:54.645367 9553 leader_election.cc:288] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 10 election: Requested vote from peers 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66:46815), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.645751 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 33 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.645864 9289 raft_consensus.cc:3032] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 LEADER]: Stepping down as leader of term 9
I20240502 20:46:54.645900 9289 raft_consensus.cc:729] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4b3cb6ad00954dc5a1270dc392870b4f, State: Running, Role: LEADER
I20240502 20:46:54.645960 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b6c379ec40417894f65f3c782c28db" candidate_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 33 } } ignore_live_leader: true dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
I20240502 20:46:54.646044 9421 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.646020 9289 consensus_queue.cc:260] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 9.34, Last appended by leader: 34, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.646150 9289 raft_consensus.cc:3037] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.646826 9421 raft_consensus.cc:2454] T a8b6c379ec40417894f65f3c782c28db P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 190c1d3968d949bbbaa27a51a2b91d8e in term 10.
I20240502 20:46:54.647055 9289 raft_consensus.cc:2396] T a8b6c379ec40417894f65f3c782c28db P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 190c1d3968d949bbbaa27a51a2b91d8e for term 10 because replica has last-logged OpId of term: 9 index: 34, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 33.
I20240502 20:46:54.647176 9491 leader_election.cc:302] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:54.647338 11578 raft_consensus.cc:2781] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 FOLLOWER]: Leader election won for term 10
I20240502 20:46:54.647394 11578 raft_consensus.cc:686] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [term 10 LEADER]: Becoming Leader. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.647482 11578 consensus_queue.cc:237] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 7.33, Last appended by leader: 33, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.650382 9289 raft_consensus.cc:1261] T fddd0fdac3ae414a8b2610752e96864b P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 8 index: 17. Preceding OpId from leader: term: 10 index: 19. (index mismatch)
I20240502 20:46:54.650424 9686 raft_consensus.cc:1261] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Refusing update from remote peer 190c1d3968d949bbbaa27a51a2b91d8e: Log matching property violated. Preceding OpId in replica: term: 9 index: 18. Preceding OpId from leader: term: 10 index: 19. (index mismatch)
I20240502 20:46:54.650730 9142 catalog_manager.cc:5484] T a8b6c379ec40417894f65f3c782c28db P 190c1d3968d949bbbaa27a51a2b91d8e reported cstate change: term changed from 9 to 10, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67). New cstate: current_term: 10 leader_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:54.650840 10554 consensus_queue.cc:1035] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20240502 20:46:54.650926 10554 consensus_queue.cc:1225] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Peer 64136bd82a30486e9d56fc28da4bd448 log is divergent from this leader: its last log entry 9.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20240502 20:46:54.650969 11532 consensus_queue.cc:1035] T fddd0fdac3ae414a8b2610752e96864b P 190c1d3968d949bbbaa27a51a2b91d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20240502 20:46:54.651598 9686 pending_rounds.cc:76] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448: Aborting all ops after (but not including) 17
I20240502 20:46:54.651645 9686 pending_rounds.cc:97] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448: Aborting uncommitted NO_OP operation due to leader change: 9.18
I20240502 20:46:54.651674 9686 raft_consensus.cc:2866] T fddd0fdac3ae414a8b2610752e96864b P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20240502 20:46:54.656035 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40118: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:54.661110 9420 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:54.661211 9420 raft_consensus.cc:482] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.661264 9420 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 11 FOLLOWER]: Advancing to term 12
I20240502 20:46:54.662101 9420 raft_consensus.cc:504] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.662668 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.662777 9553 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Advancing to term 12
I20240502 20:46:54.663516 9553 raft_consensus.cc:2454] T f6edce87fbb8469a94d38d6d2eab622c P 190c1d3968d949bbbaa27a51a2b91d8e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 12.
I20240502 20:46:54.663797 9357 leader_election.cc:302] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:54.663936 11712 raft_consensus.cc:2781] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 12 FOLLOWER]: Leader election won for term 12
I20240502 20:46:54.664006 11712 raft_consensus.cc:686] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [term 12 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.664129 11712 consensus_queue.cc:237] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.664865 9420 leader_election.cc:288] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 12 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.665285 9287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6edce87fbb8469a94d38d6d2eab622c" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.665388 9287 raft_consensus.cc:3037] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 11 FOLLOWER]: Advancing to term 12
I20240502 20:46:54.665422 9420 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:54.665518 9420 raft_consensus.cc:482] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.665567 9420 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.666182 9287 raft_consensus.cc:2396] T f6edce87fbb8469a94d38d6d2eab622c P 4b3cb6ad00954dc5a1270dc392870b4f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20240502 20:46:54.667629 9420 raft_consensus.cc:504] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
W20240502 20:46:54.668074 9249 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59728: Illegal state: replica 4b3cb6ad00954dc5a1270dc392870b4f is not leader of this config: current role FOLLOWER
I20240502 20:46:54.669371 9420 leader_election.cc:288] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 10 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.669930 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.670042 9686 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.670048 9420 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:54.670138 9420 raft_consensus.cc:482] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.670190 9420 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 11 FOLLOWER]: Advancing to term 12
I20240502 20:46:54.670961 9686 raft_consensus.cc:2454] T 2cf930db9da24ba5b1f2df86d025d376 P 64136bd82a30486e9d56fc28da4bd448 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 10.
I20240502 20:46:54.670959 9420 raft_consensus.cc:504] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.671221 9420 leader_election.cc:288] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 12 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:54.671419 9357 leader_election.cc:302] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64136bd82a30486e9d56fc28da4bd448, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:54.671949 11712 raft_consensus.cc:2781] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 10 FOLLOWER]: Leader election won for term 10
I20240502 20:46:54.672060 11712 raft_consensus.cc:686] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [term 10 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.672176 11712 consensus_queue.cc:237] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.672812 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 12 candidate_status { last_received { term: 9 index: 40 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.672915 9553 raft_consensus.cc:3032] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 LEADER]: Stepping down as leader of term 11
I20240502 20:46:54.672960 9553 raft_consensus.cc:729] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.673102 9553 consensus_queue.cc:260] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 11.41, Last appended by leader: 41, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.673234 9553 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Advancing to term 12
I20240502 20:46:54.674029 9553 raft_consensus.cc:2396] T 14d5186eb2e84ed19ab0f981067d3775 P 190c1d3968d949bbbaa27a51a2b91d8e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 12 because replica has last-logged OpId of term: 11 index: 41, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 40.
I20240502 20:46:54.674762 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5186eb2e84ed19ab0f981067d3775" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 12 candidate_status { last_received { term: 9 index: 40 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.674856 9686 raft_consensus.cc:3037] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 11 FOLLOWER]: Advancing to term 12
I20240502 20:46:54.675305 9140 catalog_manager.cc:5484] T f6edce87fbb8469a94d38d6d2eab622c P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 10 to 12, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 12 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:54.675761 9686 raft_consensus.cc:2396] T 14d5186eb2e84ed19ab0f981067d3775 P 64136bd82a30486e9d56fc28da4bd448 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 12 because replica has last-logged OpId of term: 10 index: 41, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 40.
I20240502 20:46:54.676043 9357 leader_election.cc:302] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:54.676548 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf930db9da24ba5b1f2df86d025d376" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.676637 9289 raft_consensus.cc:3037] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 9 FOLLOWER]: Advancing to term 10
I20240502 20:46:54.677513 9289 raft_consensus.cc:2396] T 2cf930db9da24ba5b1f2df86d025d376 P 4b3cb6ad00954dc5a1270dc392870b4f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20240502 20:46:54.677630 9423 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "edebc185b24441da9504c78394378f4d"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:54.677779 9423 raft_consensus.cc:482] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.677886 9423 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 11 FOLLOWER]: Advancing to term 12
I20240502 20:46:54.678515 11889 raft_consensus.cc:2726] T 14d5186eb2e84ed19ab0f981067d3775 P 88a19090795d47c2a1ea1dd979fb62bc [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20240502 20:46:54.678970 9139 catalog_manager.cc:5484] T 2cf930db9da24ba5b1f2df86d025d376 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 8 to 10, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 10 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
W20240502 20:46:54.680404 9516 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51610: Illegal state: replica 190c1d3968d949bbbaa27a51a2b91d8e is not leader of this config: current role FOLLOWER
I20240502 20:46:54.681715 9423 raft_consensus.cc:504] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.682547 9686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edebc185b24441da9504c78394378f4d" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 12 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.682647 9686 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 11 FOLLOWER]: Advancing to term 12
I20240502 20:46:54.685370 9686 raft_consensus.cc:2396] T edebc185b24441da9504c78394378f4d P 64136bd82a30486e9d56fc28da4bd448 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 12 because replica has last-logged OpId of term: 10 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 12.
I20240502 20:46:54.686383 9423 leader_election.cc:288] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 12 election: Requested vote from peers 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:54.686496 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edebc185b24441da9504c78394378f4d" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 12 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.686587 9553 raft_consensus.cc:3032] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 LEADER]: Stepping down as leader of term 11
I20240502 20:46:54.686647 9553 raft_consensus.cc:729] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.686765 9553 consensus_queue.cc:260] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.15, Last appended by leader: 15, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.686887 9553 raft_consensus.cc:3037] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Advancing to term 12
I20240502 20:46:54.687690 9553 raft_consensus.cc:2396] T edebc185b24441da9504c78394378f4d P 190c1d3968d949bbbaa27a51a2b91d8e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 12 because replica has last-logged OpId of term: 11 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 12.
I20240502 20:46:54.688202 9357 leader_election.cc:302] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88a19090795d47c2a1ea1dd979fb62bc; no voters: 190c1d3968d949bbbaa27a51a2b91d8e, 64136bd82a30486e9d56fc28da4bd448
I20240502 20:46:54.688401 11889 raft_consensus.cc:2726] T edebc185b24441da9504c78394378f4d P 88a19090795d47c2a1ea1dd979fb62bc [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20240502 20:46:54.691365 9423 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:54.691468 9423 raft_consensus.cc:482] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.691514 9423 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 11 FOLLOWER]: Advancing to term 12
I20240502 20:46:54.693096 9423 raft_consensus.cc:504] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.694705 9423 leader_election.cc:288] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 12 election: Requested vote from peers 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355), 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617)
I20240502 20:46:54.694696 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.694794 9289 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 11 FOLLOWER]: Advancing to term 12
I20240502 20:46:54.695240 9421 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3315525f1849462fbf49a285f424ea51"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:54.695351 9421 raft_consensus.cc:482] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.695400 9421 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 11 FOLLOWER]: Advancing to term 12
I20240502 20:46:54.695652 9289 raft_consensus.cc:2396] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 4b3cb6ad00954dc5a1270dc392870b4f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20240502 20:46:54.696187 9421 raft_consensus.cc:504] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.696491 9421 leader_election.cc:288] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 12 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.696779 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3315525f1849462fbf49a285f424ea51" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 12 candidate_status { last_received { term: 11 index: 43 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.696875 9289 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 11 FOLLOWER]: Advancing to term 12
I20240502 20:46:54.697643 9289 raft_consensus.cc:2454] T 3315525f1849462fbf49a285f424ea51 P 4b3cb6ad00954dc5a1270dc392870b4f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 12.
I20240502 20:46:54.697957 9355 leader_election.cc:302] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3cb6ad00954dc5a1270dc392870b4f, 88a19090795d47c2a1ea1dd979fb62bc; no voters:
I20240502 20:46:54.698127 11889 raft_consensus.cc:2781] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 12 FOLLOWER]: Leader election won for term 12
I20240502 20:46:54.698197 11889 raft_consensus.cc:686] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [term 12 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.698310 11889 consensus_queue.cc:237] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 43, Committed index: 43, Last appended: 11.43, Last appended by leader: 43, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.698632 9550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3315525f1849462fbf49a285f424ea51" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 12 candidate_status { last_received { term: 11 index: 43 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.698743 9550 raft_consensus.cc:3032] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 LEADER]: Stepping down as leader of term 11
I20240502 20:46:54.698801 9550 raft_consensus.cc:729] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.698913 9550 consensus_queue.cc:260] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 43, Committed index: 43, Last appended: 11.43, Last appended by leader: 43, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } }
I20240502 20:46:54.699059 9550 raft_consensus.cc:3037] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Advancing to term 12
I20240502 20:46:54.699903 9550 raft_consensus.cc:2454] T 3315525f1849462fbf49a285f424ea51 P 190c1d3968d949bbbaa27a51a2b91d8e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 12.
I20240502 20:46:54.700181 9139 catalog_manager.cc:5484] T 3315525f1849462fbf49a285f424ea51 P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 11 to 12, leader changed from 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 12 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } }
I20240502 20:46:54.700780 9421 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:54.700875 9421 raft_consensus.cc:482] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.700915 9421 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 11 FOLLOWER]: Advancing to term 12
W20240502 20:46:54.701071 9646 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40118: Illegal state: replica 64136bd82a30486e9d56fc28da4bd448 is not leader of this config: current role FOLLOWER
I20240502 20:46:54.701534 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b8dfbd38ea40ddb4b5c8f2b39aa43c" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.701618 9553 raft_consensus.cc:3037] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Advancing to term 12
I20240502 20:46:54.701623 9421 raft_consensus.cc:504] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.701900 9421 leader_election.cc:288] T d4e11672a55842aa92ffb09edfc7546e P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 12 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 64136bd82a30486e9d56fc28da4bd448 (127.22.45.68:42225)
I20240502 20:46:54.702553 9553 raft_consensus.cc:2454] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 190c1d3968d949bbbaa27a51a2b91d8e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a19090795d47c2a1ea1dd979fb62bc in term 12.
I20240502 20:46:54.703763 9357 leader_election.cc:302] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 190c1d3968d949bbbaa27a51a2b91d8e, 88a19090795d47c2a1ea1dd979fb62bc; no voters: 4b3cb6ad00954dc5a1270dc392870b4f
I20240502 20:46:54.704293 11712 raft_consensus.cc:2781] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 12 FOLLOWER]: Leader election won for term 12
I20240502 20:46:54.704572 11712 raft_consensus.cc:686] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [term 12 LEADER]: Becoming Leader. State: Replica: 88a19090795d47c2a1ea1dd979fb62bc, State: Running, Role: LEADER
I20240502 20:46:54.704932 11712 consensus_queue.cc:237] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } }
I20240502 20:46:54.706317 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 12 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "64136bd82a30486e9d56fc28da4bd448"
I20240502 20:46:54.706421 9688 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 11 FOLLOWER]: Advancing to term 12
I20240502 20:46:54.707401 9688 raft_consensus.cc:2396] T d4e11672a55842aa92ffb09edfc7546e P 64136bd82a30486e9d56fc28da4bd448 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20240502 20:46:54.708128 9421 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1"
dest_uuid: "88a19090795d47c2a1ea1dd979fb62bc"
from {username='slave'} at 127.0.0.1:55540
I20240502 20:46:54.708164 9553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4e11672a55842aa92ffb09edfc7546e" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 12 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.708225 9421 raft_consensus.cc:482] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 20:46:54.708247 9553 raft_consensus.cc:3032] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 LEADER]: Stepping down as leader of term 11
I20240502 20:46:54.708278 9421 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 11 FOLLOWER]: Advancing to term 12
I20240502 20:46:54.708294 9553 raft_consensus.cc:729] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.708407 9553 consensus_queue.cc:260] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "64136bd82a30486e9d56fc28da4bd448" member_type: VOTER last_known_addr { host: "127.22.45.68" port: 42225 } }
I20240502 20:46:54.708526 9553 raft_consensus.cc:3037] T d4e11672a55842aa92ffb09edfc7546e P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Advancing to term 12
I20240502 20:46:54.708986 9421 raft_consensus.cc:504] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.709275 9421 leader_election.cc:288] T 94eebae830c4437d8dc4149e7f1d8ef1 P 88a19090795d47c2a1ea1dd979fb62bc [CANDIDATE]: Term 12 election: Requested vote from peers 190c1d3968d949bbbaa27a51a2b91d8e (127.22.45.67:33617), 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65:38355)
I20240502 20:46:54.709630 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 12 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"
I20240502 20:46:54.709718 9289 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 11 FOLLOWER]: Advancing to term 12
I20240502 20:46:54.710116 9550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94eebae830c4437d8dc4149e7f1d8ef1" candidate_uuid: "88a19090795d47c2a1ea1dd979fb62bc" candidate_term: 12 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "190c1d3968d949bbbaa27a51a2b91d8e"
I20240502 20:46:54.710209 9550 raft_consensus.cc:3032] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 LEADER]: Stepping down as leader of term 11
I20240502 20:46:54.710247 9550 raft_consensus.cc:729] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 190c1d3968d949bbbaa27a51a2b91d8e, State: Running, Role: LEADER
I20240502 20:46:54.710371 9550 consensus_queue.cc:260] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.15, Last appended by leader: 15, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190c1d3968d949bbbaa27a51a2b91d8e" member_type: VOTER last_known_addr { host: "127.22.45.67" port: 33617 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f" member_type: VOTER last_known_addr { host: "127.22.45.65" port: 38355 } } peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } }
I20240502 20:46:54.710502 9550 raft_consensus.cc:3037] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 11 FOLLOWER]: Advancing to term 12
I20240502 20:46:54.710536 9289 raft_consensus.cc:2396] T 94eebae830c4437d8dc4149e7f1d8ef1 P 4b3cb6ad00954dc5a1270dc392870b4f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 12 because replica has last-logged OpId of term: 10 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 12.
I20240502 20:46:54.711310 9550 raft_consensus.cc:2396] T 94eebae830c4437d8dc4149e7f1d8ef1 P 190c1d3968d949bbbaa27a51a2b91d8e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 88a19090795d47c2a1ea1dd979fb62bc for term 12 because replica has last-logged OpId of term: 11 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 12.
I20240502 20:46:54.711560 9139 catalog_manager.cc:5484] T 61b8dfbd38ea40ddb4b5c8f2b39aa43c P 88a19090795d47c2a1ea1dd979fb62bc reported cstate change: term changed from 10 to 12, leader changed from 4b3cb6ad00954dc5a1270dc392870b4f (127.22.45.65) to 88a19090795d47c2a1ea1dd979fb62bc (127.22.45.66). New cstate: current_term: 12 leader_uuid: "88a19090795d47c2a1ea1dd979fb62bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a19090795d47c2a1ea1dd979fb62bc" member_type: VOTER last_known_addr { host: "127.22.45.66" port: 46815 } } peers { permanent_uuid: "4b3cb6ad00954dc5a1270dc392870b4f"